Open Source Integrated Library System

Evergreen on IRC

#evergreen Logs for Wednesday, February 8th, 2012

< Tuesday, February 7th, 2012Raw Log FileThursday, February 9th, 2012 >
#TimeNickMessage
#00:01:22pmplett has joined #evergreen
#00:15:13atz_ has quit IRC
#00:18:35atz__ has joined #evergreen
#00:37:01gmcharlt has quit IRC
#00:41:08gmcharlt has joined #evergreen
#01:43:20pmplett has quit IRC
#02:12:09dbwells_ has joined #evergreen
#02:15:38dbwells has quit IRC
#02:16:51pmplett has joined #evergreen
#05:43:11kivilaht1o has quit IRC
#06:16:23reosarevok has joined #evergreen
#07:45:29collum has joined #evergreen
#07:52:55kmlussier has joined #evergreen
#08:09:19akilsdonk has joined #evergreen
#08:38:56AaronZ-PLS has joined #evergreen
#08:47:52Dyrcona has joined #evergreen
#09:00:48Stompro has joined #evergreen
#09:01:41goood has joined #evergreen
#09:05:37Meliss has joined #evergreen
#09:17:15tspindler has joined #evergreen
#09:29:07jenny has joined #evergreen
#10:05:51tspindler has quit IRC
#10:07:49kivilaht1o has joined #evergreen
#10:10:49kivilaht1oHello #evergreen! I was wondering is it possible to add custom selections to PAtron's OPAC selection of "Default Hold Notification Method"?
#10:14:18jeffAre you looking to change the default preference for patrons who have never set it, or are you looking to add additional notification options -- and if so, what?
#10:20:04kivilaht1ojeff: I am looking to add additional notification options, mainly mail
#10:20:08kivilaht1ojeff: I am looking to add additional notification options, mainly mail
#10:20:13kivilaht1olike snailmail
#10:21:14kivilaht1oor in this case sending a electronic mail request to the mailing company, which will produce the physical letters and mail them away
#10:21:52tsberekivilaht1o: I hope you have a really long hold shelf period in that case.
#10:21:59kivilaht1oyeah I know it is a silly idea
#10:22:07kivilaht1obut it is part of our requiremetns specification
#10:22:14kivilaht1oits not on a must-list
#10:22:14tsbereAs for adding custom selections, yes, it is technically possible, but not easy.
#10:22:38kivilaht1oand we can use that to send bills
#10:23:04tsbereThe hold notification method options, IMO, shouldn't be used for non-hold related messages
#10:23:11kivilaht1ook
#10:23:26kivilaht1owell that covers it, thanks a lot
#10:23:32tsbereAnd if you added options for "mail me bills" everyone would deselect it, I think ;)
#10:23:48kivilaht1oyeah
#10:23:52kivilaht1omaybe we just mail them
#10:23:59tsbereThat is what we do
#10:24:06tsbereBilling notices aren't emailed, only snail mailed
#10:26:22jeffkivilaht1o: as tsbere touched on, you can add additional hold notification methods with some code/development work.
#10:26:54jeffwe stopped sending postal notices for holds when we migrated to evergreen. email and automated phone calls are the two options we offer.
#10:31:17berick grabs 0675
#10:31:34hopkinsju has joined #evergreen
#10:38:22kivilaht1ojeff: I am also interested about work estimates for tasks such as adding a new notification option
#10:38:42kivilaht1oyou could say I am a kind of techie ;)
#10:39:07kivilaht1oILS and Perl stuff was just new to me 8 months ago
#10:39:31tsberekivilaht1o: The Staff Client, Opac, some backend code, and a database table would all need changes. Some more so than others.
#10:39:51tsbereThe database table because each hold notification option is at least one column in the hold_request table
#10:40:22kivilaht1obut the default notification method is stored in just one field
#10:40:31tsbereThat is stored in a user preference, yes.
#10:40:49kivilaht1obut how to deal with the extra variables takes effort
#10:41:02kivilaht1oyeah I get that
#10:41:55kivilaht1obut isnt it just about changin the Action Trigger Reactor?
#10:43:25hopkinsju has quit IRC
#10:44:05tsberekivilaht1o: Technically there are (in master) 2 action triggers involved by default. And neither touches phone calling. One is for emails, the other for SMS.
#10:44:35denialsberick: for the vandelay sluggishness, I wonder if a database with large number of bibs results in slowness due to match attempts (particularly with missing indexes)
#10:45:02tsberekivilaht1o: And *neither* of those uses the user's default preference as a decision point. The hold itself holds all the details. I have a branch to make it more intelligent about it, but the user's default preference is only ever consulted when the hold is first placed.
#10:47:35berickdenials: yeah, probably so
#10:48:31hopkinsju has joined #evergreen
#10:50:44kivilaht1otsbere: is the SMS Reactor for 2.2 as I dont find it in 2.1?
#10:50:57kivilaht1oI can mark that off of our todo list
#10:52:52Dyrconakivilaht1o: I can answer for tsbere: the SMS reactor is for 2.2.
#10:52:56mrpeters-isli can go through and set the date/time on our 20+ virtual machines EVERY day, make sure ntp is running, timezone is right
#10:52:57berickdenials: i can set up a test on a larger system (40k records) w/ liberal matching later this week and see how it behaves..
#10:53:02mrpeters-isl2 days later, the time will be off by hours
#10:53:22mrpeters-islwhat the hell is going on? it's making log digging impossible.
#10:53:39kmlussierdenials, berick: I checked with MVLC catalogers shortly after I heard about the vandelay sluggishness, and they weren't seeing the same performance problems with a large number of bibs.
#10:54:03denials vaguely remembers adding an index to address vandelay matching sluggishness
#10:54:04kmlussierThe sluggishness seemed to be more pronounced with 020 matching.
#10:56:17tsberemrpeters-isl: I find that NTP works wonderfully if the service is running....provided it can talk to the server it was configured to use. What server is it using?
#10:57:21mrpeters-isltsbere: it seems to just stop at some point
#10:57:28mrpeters-isli'm connected to several right now, and syncd with them
#10:57:42mrpeters-islbut ill come back in a few days and this machine, and all of it's VM's will be off by over 2 hours
#10:58:14jefftime sync in a virtual environment can be quite problematic, though it has gotten better in the last 5 years or so. there are likely specific recommendations for the virtualization environment you're in, and it's important to seek them out and at least give them some testing.
#10:58:37mrpeters-isljeff: sure, but we're talking hours and hours over a matter of a day or two
#10:58:42jeffthe recommendations can change from version to version, or flavor to flavor.
#10:58:45mrpeters-island the non-virtual machines suffer the same fate
#10:59:23mrpeters-islperhaps ill just restart ntp nightly and see if that helps
#10:59:46jeffhow are the non-virtual machines configured? how many peers do they have, and are all of the peers off-site, or are they peering with each other, or with any of the virtual machines?
#11:00:40tsberemrpeters-isl: Some virtual machines run their clocks on the host machine's clock. Others run their own clock independant of the host's. It is important to know *which* kind of VM you are running. Running NTP on the VM when the VM is based on the host's clock will actually screw up *faster*.
#11:01:03jeffcan you paste output of "ntpq -c lpeer" and "ntpq -c readlist" on a physical host which is showing symptoms of being off by many minutes?
#11:01:45mrpeters-islntpq: read: Connection refused --- which, of course, is because NTP process has died off
#11:02:04jeffcheck logs (say, daemon.log) for news of why ntpd isn't running?
#11:03:07mrpeters-islhttp://pastie.org/3342106
#11:03:40jeffthat's about what i'd expect for a fresh start. nothing much to observe there.
#11:03:47jeffany news from the logs as to why ntpd wasn't running?
#11:04:31mrpeters-islnothing. ill just try restarting it nightly and see if that improves.
#11:04:53jeffwhen you say "the non-virtual machines suffer the same fate", are you speaking exclusively of physical hosts which are themselves running virtual guests?
#11:04:58mrpeters-islright
#11:05:14mrpeters-islso im sure the guests are getting their time from the host
#11:05:15jeffah. initially i thought you meant machines not involved in any way with running VMs.
#11:05:23mrpeters-islno, no, sorry
#11:05:36mrpeters-islill get it figured out. thanks for the advice.
#11:05:48tsberemrpeters-isl: I point back to the point of "how are the virtual machines running their clocks"
#11:06:10tsberemrpeters-isl: If both the VM and the host machine are using the same "clock" and you have NTP running on both they will fight until NTP gives up on both, I think.
#11:06:30mrpeters-islyeah, ill look into how they are getting their time.
#11:06:34jeffxen.independent_wallclock was a sysctl which i believe went away if you're using paravirtualization.
#11:06:39mrpeters-islsure isn't helping that debian's indiana time zone is broken
#11:07:08Stompromrpeters-isl: I've seen this problem also, in my case it was related to cpu frequency scaling on the host. The vm's don't understand that the host's processors frequency can vary, so their clocks speed up/slow down based on the system load of the host. I just disable the frequency scaling of the host to fix.
#11:07:26mrpeters-islthanks, Stompro
#11:07:26tsbereOur VMs have independent clocks, so I run ntp on the VM and the host. But on my windows-based VM host (my desktop) the opposite is true, they use the host machine clock, so ntp is only done on the host.
#11:08:25jeffmrpeters-isl: are you guys running Xen 4?
#11:08:29mrpeters-islya
#11:47:26atz__ has quit IRC
#12:06:43atz has joined #evergreen
#12:22:49moodaepo has joined #evergreen
#12:25:57AaronZ-PLS has quit IRC
#12:25:57goood has quit IRC
#12:26:31goood has joined #evergreen
#12:28:46goood has quit IRC
#12:42:23moodaepo has quit IRC
#12:42:32moodaepo has joined #evergreen
#12:42:39moodaepo has joined #evergreen
#12:51:33mieke has joined #evergreen
#12:52:20mieke has quit IRC
#13:02:21mrpeters-islanyone on 2.1 using a socks 5 proxy for the staff client? I don't think http://open-ils.org/dokuwiki/doku.php?id=evergreen-user:configuring_a_proxy_for_the_staff_client works anymore.
#13:02:53mrpeters-islmy proxy is working (can use it for HTTP) and is configured in all.js but xulrunner can't seem to use the proxy
#13:02:56tsbereI am not on 2.1, I am not using said proxy, I can still probably help figure it out
#13:03:30mrpeters-islim just wondering if something changed with the later xulrunner
#13:03:47moodaepo has quit IRC
#13:03:53mrpeters-islabout:config is configured with localhost 9999 and socks 5
#13:03:55mrpeters-islas is all.js
#13:04:03tsberemrpeters-isl: Well, one thing I *highly* disagree with is changing that all.js file
#13:04:05moodaepo has joined #evergreen
#13:04:12mrpeters-islyeah?
#13:04:51tsbereThat will break on a regular basis if upgrading xulrunner unless you keep editing it. Adding a new file with the proper settings to the defaults/preferences folder or just having people set things a little more manually (via about:config) would be better
#13:04:58mrpeters-islill uninstall and try with a fresh slate
#13:06:26tsberemrpeters-isl: Would be somewhat trivial, for example, to update the windows installer code for "if this file exists, offer proxy config as an option" for the staff client. Then drop the proxy config into a file in a customization branch and when you build staff clients the proxy config gets loaded by default.
#13:07:29tsbere considers editing that page to add a "DO NOT EVER DO THIS IN THIS MANNER" warning to that section of that page
#13:08:04mrpeters-isl:)
#13:08:22mrpeters-islso, how would one access the staff client to get to about:config if they ONLY had access via a proxy
#13:08:26mrpeters-islyou'd have no way to configure it, right?
#13:08:32tsbereThere are two options
#13:08:36tsbere1 - Edits prefs.js
#13:08:43tsbere2 - Add a file to the default preferences set
#13:10:03mrpeters-isli guess i have no idea what to add to prefs.js
#13:12:31mrpeters-isli should say...i wouldn't if i didn't have access to about:config
#13:12:49tsberemrpeters-isl: I will add a better section to that page in a moment.
#13:13:00mrpeters-islbut, alas, still wont use my proxy
#13:13:35mrpeters-isli think it tries...but fails
#13:14:24mrpeters-islhttp://pastie.org/3342899
#13:14:28goood has joined #evergreen
#13:14:52mrpeters-islshould do the trick, my proxy is listening on 9999. i can use it for other things (web, sockscap, etc.) so it seems to just be isolated to xulrunner
#13:15:57mrpeters-islill just sockscap the staff client. that makes it work fine.
#13:17:47tsberemrpeters-isl: As I said, I am adding a better section to the page
#13:18:04mrpeters-isli know. ill have a look. i've got my workaround for now.
#13:23:09atz_ has joined #evergreen
#13:23:27tspindler has joined #evergreen
#13:24:07moodaepo has quit IRC
#13:25:09moodaepo has joined #evergreen
#13:25:38atz has quit IRC
#13:27:18tsberemrpeters-isl: Updated.
#13:27:29tsbere wonders what people might think of his edit to the original section
#13:29:41csharpI'm seeing a lot of this in our logs - any clues as to the source?: http://paste.lisp.org/display/127593
#13:30:02csharpjust asking in case someone "just knows"
#13:30:55tsberemrpeters-isl: I think your issue is the proxy type being 5 instead of 2, for the record.
#13:31:07graced has joined #evergreen
#13:32:17tsberecsharp: Not sure....do you have anything on the IP in question that is communicating to the system, or is the system trying to communicate the other way?
#13:34:08csharptsbere: not sure about that... will investigate
#13:34:09csharpthanks
#13:35:08collum has quit IRC
#13:37:37atz_ has quit IRC
#13:40:54atz_ has joined #evergreen
#13:42:17goood2 has joined #evergreen
#13:42:21goood has quit IRC
#13:43:16collum has joined #evergreen
#13:57:05eeevilkmlussier: the "only slow for 020 matching" is a good clue, I think ... most likely a missing index, as dbs suggested. specifically one on metabib.real_full_rec.value where tag='020' and subfield='a'. note, 2.2/master has (your, IIRC) vandelay match set stuff, which works differently
#13:57:14bshumtsbere++ #big red AVOID message :P
#13:58:22kmlussiereeevil: Thanks! I'll pass it along.
#14:03:47AaronZ-PLS has joined #evergreen
#14:03:56hopkinsju has quit IRC
#14:07:42hopkinsju has joined #evergreen
#14:15:31edoceoHey, I just got that osrf-http-translator thing working - that's awsome ! So, now how to I find a list of all the APIs I can call? grep the code ?
#14:19:17yboston has joined #evergreen
#14:19:46tsbereedoceo: You should be limited to methods on services published on the "public" side. You can get that list from the config files.
#14:20:04tsbereYou can use srfsh and the "introspect" command to find out what those services provide for commands
#14:20:15denialsedoceo: have you read the various opensrf docs / tutorials?
#14:20:41edoceoI've been through many of them; all over the evergreen site and some others
#14:20:43denialshttp://bzr.coffeecode.net/2011/tpac_tutorial/Evergreen_development_101.html might be useful
#14:20:48edoceoYep
#14:21:01atz_ has quit IRC
#14:21:25atz_ has joined #evergreen
#14:21:27edoceoI spent most of Tuesday reading; not sure I got 100% retention; so now I'm hacking some code to try to hammer that stuff in there
#14:22:09denialsedoceo: right, so "introspect" is written up at "Getting documentation for OpenSRF methods from the srfsh client" in http://journal.code4lib.org/articles/3284 for example.
#14:22:53denials notes that the docs that are floating around are definitely not the be-all / end-all, but they're a start
#14:23:57edoceoAnd I can get the list of services from the IDL or something?
#14:24:15edoceoor perhaps the opnesrf.xxml - or one of these config files?
#14:24:46Dyrconaedoceo: There is a program called srfsh. You run it. You type introspect open-ils.cat, and you get a list of all the open-ils.cat methods, and so on.
#14:24:54edoceoI got that
#14:25:08edoceo<appname>
#14:25:12DyrconaThere is no list of APIs apart from that and the code.
#14:25:22edoceoI will make one
#14:25:40atz_ has quit IRC
#14:25:40Dyrconaedoeo: It will be obsolete before you're done. :)
#14:25:58edoceoI'll make new ones every day! :D
#14:26:00denialsDyrcona: well, there's docgen.xsl
#14:27:39atz_ has joined #evergreen
#14:28:01bshumI was just looking that up too and annoyed that it wasn't on our production server side :(
#14:28:06atz has joined #evergreen
#14:28:31jennyFYI folks, the Evergreen Reports Taskforce meeting will be starting here in about 2 minutes.
#14:28:35denialshttp://evergreen-ils.org/blog/?p=46 with a working example at http://dev198.esilibrary.com/opac/extras/docgen.xsl?service=open-ils.search&all=on&method=opensrf.system.method.all
#14:28:59denials shuts up
#14:30:07kmlussiereeevil: I'm seeing 020a entries in metabib.real_full_rec, so I don't think that's the problem.
#14:30:37jennyTime to start the Evergreen Reports Taskforce meeting...
#14:31:08jenny#startmeeting Evergreen Reports Taskforce. The agenda may be found here: http://open-ils.org/dokuwiki/doku.php?id=evergreen-reports:meetings:2012-2-08-agenda
#14:31:08pinesol_greenMeeting started Wed Feb 8 14:31:08 2012 US/Eastern. The chair is jenny. Information about MeetBot at http://wiki.debian.org/MeetBot.
#14:31:08pinesol_greenUseful Commands: #action #agreed #help #info #idea #link #topic.
#14:31:49moodaepojenny+=
#14:31:49jenny#topic Introductions
#14:31:53moodaepojenny++
#14:32:28jenny is Jenny Turner, PALS (and our notetaker will be MeetBot, if I remember to use the commands!)
#14:32:31atz_ has quit IRC
#14:33:02tspindler is Tim Spindler - CW MARS
#14:34:40jennytspindler: is it just you and me for the reports group!? Everyone else must be at Code4Lib... :)
#14:35:11tspindlerjenny: may be
#14:35:41Tony_ has joined #evergreen
#14:35:57Tony_Hello everyone
#14:36:13tspindlerjenny: who was going to talk about Jasper again?
#14:36:34jennyjeff
#14:37:18Tony_New to this, please pardon my inexperience...
#14:37:34jennyTony_: hi - are you here for the Reports Taskforce meeting or for general Evergreen assistance?
#14:37:56Tony_general assistance...looking for Mac client info...is there another channel?
#14:38:28jennyNo - we were just trying to hold a meeting, but it appears to be sparsely attended! We'll probably wrap up pretty soon...
#14:38:55Tony_ok..I will exit and try later...I apologize
#14:38:57jennytspindler: should we cancel for the month? Attempt to reschedule for later in the month? What are your thoughts
#14:39:08jennyTony_: no worries!
#14:39:24tspindlerjenny: that makes the most since since it looks like its just us
#14:39:37jennyok
#14:40:14jenny#info Due to low attendance, the February Reports meeting is postponed
#14:40:27jennytspindler: thanks for coming!
#14:40:32tspindlernp
#14:40:39jenny#endmeeting
#14:40:39pinesol_greenMeeting ended Wed Feb 8 14:40:39 2012 US/Eastern. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
#14:40:39pinesol_greenMinutes: http://evergreen-ils.org/meetings/evergreen/2012/evergreen.2012-02-08-14.31.html
#14:40:39pinesol_greenMinutes (text): http://evergreen-ils.org/meetings/evergreen/2012/evergreen.2012-02-08-14.31.txt
#14:40:39pinesol_greenLog: http://evergreen-ils.org/meetings/evergreen/2012/evergreen.2012-02-08-14.31.log.html
#14:40:49Dyrcona sent Tony_ messages in private chat.
#14:41:25jennyDyrcona++
#14:41:38mrpeters-islanyone done a conversion from Concourse > Evergreen?
#14:42:12Dyrconawow. short meeting.
#14:42:13goood2 has quit IRC
#14:42:14goood has joined #evergreen
#14:43:16Tony_Thank you...bye
#14:43:46jennyMy record is 8 minutes. :)
#14:43:50Tony_ has left #evergreen
#14:47:36b_bonner has joined #evergreen
#14:48:27mieke has joined #evergreen
#14:49:55joannet has joined #evergreen
#14:52:53atznice bot tools
#14:55:25eeevilkmlussier: sorry, no, not missing data, a missing (sql-level) index on the data to make isbn matching faster
#14:58:37Dyrcona wishes *they* would actually read and pay attention to what is on the screen, otherwise known as their job.
#14:58:46mieke has quit IRC
#15:00:13eeevilkmlussier: something along the lines of: CREATE INDEX CONCURRENTLY full_rec_isbn_idx ON metabib.real_full_rec (value) WHERE tag = '020' and subfield = 'a';
#15:00:24enhancin has joined #evergreen
#15:00:59goood has quit IRC
#15:01:08goood has joined #evergreen
#15:01:09enhancinIs there a way to hve the Marc Batch Import to be multithreaded?
#15:01:15enhancinor would I have to do that through the command line
#15:01:57Dyrconaenhancin: To what MARC Batch Import do you refer? Vandelay, in the client, or something else?
#15:02:07enhancinah, in the client, sorry
#15:02:09enhancinthe staff client
#15:02:21enhancingopt about 120,000 records to import
#15:02:29DyrconaThe answers to your questions are No and Yes respectively.
#15:02:40enhancinAh, alright. thanks.
#15:02:49enhancini guess it's going to be worth the time saved in the end =P
#15:02:54Dyrcona120,000 records should be sufficient to kill Vandelay.
#15:03:22enhancinshould I chunk it then/
#15:03:32kmlussiereeevil: heh, this time I really will pass it along since you've just lost me. But I'm sure it will be helpful to someone here. Thanks again!
#15:04:17Dyrconaenhancin: Look in Open-ILS/src/extras/import in an Evergreen git checkout or expanded tarball.
#15:04:59Dyrconaenhancin: I do not use Vandelay, but our catalogers do. I would not dream of loading that many records with Vandelay, even chunked.
#15:05:07enhancinhttp://docs.evergreen-ils.org/2.1/html/migrating_records_using_migration_tools.html ?
#15:05:16enhancini think that's the doc page that talks about those scripts
#15:05:20DyrconaThat might help.
#15:05:24enhancincool
#15:05:32enhancindone that process before on another server : )
#15:06:18Dyrcona writes his own tools for mass loading of bibliographic records.
#15:07:13DyrconaWhich reminds me that I should be figuring out what to do with our records from Backstage.
#15:08:37kmlussierDyrcona: The authority records?
#15:08:54Dyrconakmlussier: authorities and updated bib records.
#15:09:23DyrconaIt is apparently not as simple as just overlaying everything, since our catalogers seem to edit records after they are sent to Backstage.
#15:09:36joannet:quit
#15:09:41joannet has quit IRC
#15:09:58kmlussierI tried uploading them a couple of times through Vandelay, but I've never been able to get the authority match sets to work.
#15:10:24Dyrcona nods slowly in recognition.
#15:11:01kmlussierhttps://bugs.launchpad.net/evergreen/+bug/867691
#15:11:01pinesol_greenLaunchpad bug 867691 in Evergreen "Authority match sets are not working" (affected: 1, heat: 1) [Undecided,New]
#15:11:07DyrconaOur authorities, at least the one imported from Horizon, but likely all of them, are a mess.
#15:11:52DyrconaI had to throw out well over half of our authority records during migration because they were not even valid MARC when exported from Horizon.
#15:12:05DyrconaToo bad youdonotexist is not here.
#15:12:40kmlussierI'm still not clear on the purpose of the updated bib records since Evergreen can update the linked bib records.
#15:12:52kmlussierOr can it?
#15:13:09Dyrcona isn't clear about the purpose of us using such a service in the first place.
#15:13:29DyrconaThe authorities we get from backstage are not linked to anything, afaict.
#15:15:20DyrconaI plan to write something to load the records both authorities and bibs.
#15:15:38DyrconaI'll need to check dates on the 005 in bibs and not overlay anything that is newer in our database.
#15:15:49DyrconaStill not sure what to do about/with authorities, though.
#15:20:23atz has quit IRC
#15:21:09enhancinDyrcona, I've got two files here one for the biblio records and one for the authority records...could I do the auth as a batch import after I import these biblios via command line or will I also have to use this method for the auth as well? (currently converting biblio to bre format...)
#15:21:34Dyrcona points at this previous in-channel statement.
#15:21:46enhancinlolol
#15:22:18enhancinwhen i did a tst of 500 the biblio took a long time but the auth was really quick
#15:22:28enhancinso that's why i'm wondering if it would be okay to just do it through the staff client for the auth records.
#15:22:55Dyrconaenhancin: sure. it the performance is acceptable.
#15:23:05Dyrcona^it^if
#15:23:46enhancinI hope so, it's got 2.4GHz cores on it..single core should be fine to process auth records
#15:24:58Dyrconaclient or the server(s)?
#15:25:09enhancinthe server, that's what processes the auth records, right?
#15:25:12DyrconaI think vandelay is more intensive on the backend.
#15:25:47DyrconaEvergreen doesn't do as much with authorities as it does with bibs.
#15:25:54enhancinright
#15:25:57Dyrconathe database triggers are simpler, etc.
#15:26:03enhancinthat's why i'm hoping it should be fine via that form for the auth records
#15:26:08enhancinup to a limit...not sure what the limit is
#15:26:26DyrconaI think that limit depends on many factors.
#15:26:37enhancinyeah.,
#15:27:04edoceoIs it required to run Evergreein over SSL? Can I disable that and only go plain-text? Makes it easier for me to sniff traffic
#15:27:17DyrconaI've heard reports of people having problems loading just 2,000 records via vandelay on a powerful, but busy server.
#15:27:30enhancinah
#15:27:36enhancinserver isn't very busy especially over night
#15:27:50Dyrconayep.
#15:28:15DyrconaI've loaded 10,000 on a "server" on a desktop machine overnight, but all it was doing was loading the records.
#15:28:36enhancinyeah. we've done a 5,000 i think before...i can't remember what I did last timne but I did it through the staff client and it worked fine
#15:29:10enhancinoh woah, 23,000 last time
#15:29:41Dyrconasome of it depends on patience and thing not going horribly wrong in the middle.
#15:30:41enhancinyeah, it wasn't busy, i had converted it beforehand to make sure it was in the good format and it really didn't take too long
#15:31:30Dyrconaedoceo: you could attempt to disable ssl, but I think you will find it is quite a bit of work.
#15:34:31tsbereedoceo: Why do you want to sniff traffic?
#15:35:49jeffedoceo: If possible, rather than attempt to disable SSL, I'd recommend using an SSL proxy that's designed for development / debugging.
#15:36:24jeffedoceo: I've found that in most cases I can do what I need to do with a little client-side debugging, though -- so I don't have a go-to proxy to recommend.
#15:36:56jeffedoceo: IMO, better to work with SSL/TLS on from the start than to try and put it "back" later. :-)
#15:37:21jeffall depends on what you're doing, though. my advice may not be suitable for you.
#15:38:08edoceotsbere: watch requests to the /osrf-http-translator
#15:38:18jeffedoceo: also, if you're trying to get some visibility into the staff client's operations and API calls, I highly recommend enabling the console and turning up the default debug level. You can see the API calls being made, their arguments, and the objects returned.
#15:38:24edoceojeff: do you know of a proxy like tool I could put in front of my apache?
#15:38:38edoceoOh, yea - console
#15:39:12tsbereedoceo: I am not sure why you need to watch those requests....what is your overall goal?
#15:39:26jeffCharles and nginx both come to mind as being possible options, but as I mentioned -- I haven't found myself needing them for that specific purpose lately, thus can't make a strong recommendation.
#15:39:28edoceoLook at API requests to the osrf-http-translator
#15:40:19tsbereedoceo: Let me put it a different way: What functionality are you attempting to duplicate, as I am assuming that you are trying to avoid using the staff client for some reason on some task?
#15:41:12edoceonone, yet; I'm not trying to avoid the staff client
#15:41:32edoceoMy goal here is simple: look at those requests/responses "on the wire"
#15:41:42eeevilkmlussier: a belated NP ;)
#15:41:57Dyrcona eeevil is NP complete.....
#15:42:26edoceoI want to know more about what is going on in the guts of evergreen; so I'd like to watch that part and I'm also heavily logging the SQL
#15:42:32berickedoceo: you can everything coming in and out of the translator in the gateway log w/ sufficient log level
#15:42:34tsbereedoceo: Still not sure why you need to look at things "on the wire" - If you are looking at your own stuff, just capture it in your code. If you are looking at something else's code, what code? :P
#15:42:55bericks/you can/you can see/
#15:42:56jeffberick++
#15:43:06Dyrconayeah, turn the logging all the way up to elebentee!--Hope you have infinite storage!
#15:43:14berickhaha
#15:43:26edoceolimited to 5TB
#15:43:26jeffI suggested cranking client-side debug levels / console, but server side is also an option, if you have both access and a sufficiently not-production server. :-)
#15:43:49edoceoyea, my environment is hack-a-tronic
#15:43:59jeffsuperb.
#15:44:00DyrconaFair warning: My *dev* machine filled the disk over a weekend with logging all the way up, and it has 80 GB of space, with about 40 of that free, typically.
#15:44:26berickindeed, crank it up, poke around, then turn it down (or don't use it)
#15:44:27jeffyeah, do not leave debug logging unattended. ;-)
#15:47:10kmlussier has left #evergreen
#15:50:45atz has joined #evergreen
#15:56:07yboston has quit IRC
#15:57:36tspindler has quit IRC
#15:57:46Meliss has quit IRC
#15:59:16Dyrcona grumbles: 5TB! Why in my day, we had 360k per side and we liked it! :)
#16:00:09sylvar has quit IRC
#16:00:40sylvar has joined #evergreen
#16:01:18goood2 has joined #evergreen
#16:01:18goood has quit IRC
#16:09:38collum has quit IRC
#16:14:55akilsdonk has quit IRC
#16:21:52moonburn has joined #evergreen
#16:23:42moonburnAny idea what would cause the birthday with 00 for months under Patron, and checkout dates showing 59 for the month under Checkout tab under Patron?
#16:25:32tsberemoonburn: What is your local date/time format?
#16:26:16moonburnAs far as `date` on the server you mean?
#16:26:25tsbereI was thinking workstation
#16:26:52moonburnwell, i'm EST, the server is MST
#16:31:07vxbush has joined #evergreen
#16:32:21vxbushRandom odd question of the day: Trying to start Apache on a system where I've installed 2.1.1 (continuing the CentOS love of last week) and I'm getting a weird error:
#16:32:35vxbushhttpd: Syntax error on line 1012 of /etc/httpd/conf/httpd.conf: Cannot load /usr/lib/httpd/modules/osrf_json_gateway.so into server: libopensrf.so.2: cannot open shared object file: Permission denied
#16:32:40vxbushWhat does it want permission for?
#16:33:10DyrconaIt wants permission to read and load libopensrf.so.2.
#16:33:37vxbushThat file is owned by opensrf and everyone has read/write privileges. But that doesn't seem to be good enough.
#16:34:00Dyrconawell, you don't want read/write, just write.
#16:34:07Dyrconajust read.... doh.
#16:34:09vxbushOkay, trying that......
#16:34:27Dyrcona has made a file write-only on purpose once.
#16:34:43vxbushBut everything else has read/execute permissions.
#16:34:46tsberevxbush: I bet it is the "you can't execute things everywhere" crap. Disable SELinux and try again?
#16:34:49vxbusheverything else in that directory, I mean
#16:35:00Dyrconaah, SELinux......
#16:35:20Dyrconathan you, National Security Agency for making our computers so secure that we can't use them.
#16:35:26vxbush[vx steps aside to deal with something in meatspace]
#16:35:36moodaepocazzerson++
#16:36:01Dyrcona sticks Debian and derivatives, so never deals with SELinux.... has forgotten the pain.
#16:36:11Dyrcona apparently can't type either.
#16:36:18DyrconaMust be time to go home.
#16:36:39moodaepoOops wrong window aye
#16:37:03atz has left #evergreen
#16:37:17bericktsbere: did you ever track down the problem you were having with open-ils.cat dying (or am I misremembering the problem)?
#16:37:27tsbereberick: I thought so. I was wrong.
#16:37:28tsbere<_<
#16:37:43tsbereI have theories, but none have panned out yet. Most recent crash met none of them.
#16:38:08bericktsbere: ah, k. for you, does the listener process die?
#16:38:29moonburntsbere: any ideas?
#16:39:01tsbereberick: Router and listener (and drones!) are all still there, no runaway CPU usage, etc. I suspect that either the router is getting a stream error from XMPP (as in, not a message stanza) and fails to realize it has gotten that, or that the listener process is waiting on something that will never return.
#16:39:26tsberemoonburn: My first guess is "your date/time formats are wonky" - A lot of those base on the workstation's date/time format.
#16:39:49tsberemoonburn: Beyond that, I stop guessing and leave you to your own devices.
#16:40:08gmcharltberick: most recently been seeing same thing as tsbere - listener continues to run, but gets disconnected from ejabberd
#16:40:31tsberegmcharlt: I didn't see it disconnected from ejabberd. I checked, the resource was still active and accepting messages.
#16:40:51tsberewithout being redirected to a drone instead, etc
#16:41:09gmcharltyeah, my observation today was that both the router and the listener for open-ils.cat had dropped off the connected_users list
#16:41:25gmcharlteven though the processes were still alive
#16:41:27tsbere*that* I saw after we restarted everything. Within 2 min of issues.
#16:41:53tsbereBut the router likely dropped the listener when the listener's connection dropped
#16:42:12tsbere walked through that code recently to see where things were going wrong
#16:42:42gmcharltcorresponded to a 'Timeout on gen_tcp:send' in ejabberd.log before the listener session was closed
#16:42:43tsbere found little to nothing on that front, but suspects the problem is in the listener more than the router at this point
#16:43:01tsberegmcharlt: We don't even get that much for most of our issues. :(
#16:44:00tsberegmcharlt: Just on the "just after we restarted all services" open-ils.cat death. All others no abnormal disconnects. :/
#16:44:28berickhmm, open-ils.cat listener inbound network buffer filling up, perhaps (i.e cat waiting on something that won't return)
#16:44:50bericktsbere: are you running latest opensrf?
#16:45:20tsbereberick: Close to. We have the lock file stuff in.
#16:45:40berickk
#16:46:06tsbereWe don't have the JSON_v0 stuff, basically. If that *is* causing the issue I am amazed that client side javascript is causing server side megafails. <_<
#16:46:20berickno, more likely the lockfile
#16:46:41berick(or something else entirely, of course)
#16:46:44tsbere even ran through a list of "what could cause this?" speculation that resulted in didlysquat so far other than mindgames
#16:47:16DyrconaIf it is the lockfile changes, I'd expect to see it on our dev servers and it never seems to happen there, even when we "try."
#16:47:31tsbereI thought "perhaps the router is getting an error stanza due to max message size when the message gets *bigger* due to the listener having a longer name, thus assuming the listener is dead and removing it, and then removing the entire service". That wasn't it, according to what I saw today.
#16:47:51vxbushOkay, it turns out I don't have enough power to turn off the selinux stuff. I'm going to have to ask a more powerful admin to deal with that. I did remove the write permissions on the libopensrf file, and that alone didn't fix it.
#16:48:20tsbereberick: Basically, I am out of ideas for it being the router. I am now at "the listener is going into an infinite wait".....
#16:48:54tsbereberick: What would happen if the drone failed to read all of the data on the pipe and, say, crashed while reading?
#16:49:05berickfrom what i've heard so far, that'd be my first guess too
#16:49:40bericktsbere: the parent has a dedicated pipe to each child. if a drone dies, the parent would reap it and move on
#16:51:02tsbereberick: what about the lock file?
#16:51:22Dyrcona grumbles about pipes and SIGPIPE for no damend reason on Ubuntu 10.04.....
#16:52:42bericktsbere: it is a shared lockfile for all. if a child process dies while locking, the lock should clear.
#16:53:07berickif either parent or child had problem locking/unlocking it should log an error as well
#16:53:35bericks/shared lockfile for all/shared lockfile for the listener and its drones/
#16:54:13berickgmcharlt: have you by any chance captured any strace output of the listener process while it's frozen?
#16:55:00jeffstrace++
#16:56:39tsbereberick: You know, that is a good idea. I should make sure I do that myself the next time this happens. >_>
#16:58:32tsbere installs strace for that purpose....it wasn't there
#17:01:08goood2 has quit IRC
#17:01:17goood has joined #evergreen
#17:02:27AaronZ-PLS has quit IRC
#17:19:44gmcharltberick: no, but I'll keep that in mind next time
#17:24:19moodaepo has quit IRC
#17:24:25Dyrcona screams as he runs from the room.
#17:24:27vxbush has quit IRC
#17:24:32Dyrcona has quit IRC
#17:31:26moonburn has quit IRC
#17:36:51moodaepo has joined #evergreen
#17:36:51pmplett has quit IRC
#17:40:28goood2 has joined #evergreen
#17:40:28goood has quit IRC
#17:43:27jenny has quit IRC
#17:53:55moodaepo has quit IRC
#18:09:43youdonotexist has joined #evergreen
#18:25:33edoceoI don't suppose there is a way to get a list of all assets/copies in evergreen? I suppose I could just call, in sequence, open-ils.search.asset.copy.retrieve ?
#18:27:58mtcarlson has joined #evergreen
#18:28:00bshumThat sounds like it would take a long time. With lots of potential gaps depending on how data was migrated / created.
#18:28:50edoceoI'm open to suggestions on how to index the whole list of assets
#18:29:19hopkinsju has quit IRC
#18:29:54edoceoBut, failing any magic; I might just brute-force with a loop on open-ils.search.biblio.record.mods_slim.retrieve
#18:32:36bshumedoceo: Out of curiosity, what's your objective with using open-ils function calls to retrieve all copy data in that manner? vs. like… running a report / giant SQL
#18:33:13edoceoobjective: play with opensrf/open-ils
#18:33:31bshumedoceo: Ah, fun objective :P
#18:33:37edoceoBut yea, indexing would be easier to just maybe pump everything into a big ts_vector thing in PG
#18:34:09edoceoYea, I just built a little play-kit that allows me to do a search with a PHP front end - cute!
#18:34:16bshumwell, it just seems… painful, to be retrieving "all assets/copies" in a system. Unless it were a smallish organization.
#18:34:26bshum doesn't want to imagine a report listing every item in our consortium........
#18:35:08edoceoHow do those discovery products work on Evergreen? to they direct query the SQL and then, maybe import to Lucene or somethign?
#18:35:25tsberebshum: I ran one of those once. With copy counts and even returned deleted items. Postgres choked. <_<
#18:36:16bshumedoceo: I'm not an expert (i could try asking one around c4l conference crew), but I believe they export data and import them into other databases on a nightly basis.
#18:36:31edoceooh, hmm
#18:36:33bshummoodaepo's org runs VuFind in that manner.
#18:36:54bshumWhere they export their Evergreen data and then add it to some Solr index or some such.
#18:37:12bshumtsbere: Ugh, that sounds… nasty.
#18:37:46tsberebshum: Was an experiment, technically before we migrated. New DB server might be able to handle it.
#18:37:52edoceoOh, but I wonder, could I perhaps just use some API to see all acquisitions for the day to index? Or perhaps I could attach an event listener someplace....
#18:40:08mtcarlson has quit IRC
#18:41:36graced has quit IRC
#18:48:41mtcarlson has joined #evergreen
#19:01:41moonburn has joined #evergreen
#19:03:30mtcarlson has quit IRC
#19:05:15mtcarlson has joined #evergreen
#19:07:15jeffedoceo: yes. there are a number of ways to do that with tweaks to supercat or with database triggers.
#19:07:22jeffno examples handy at the moment.
#19:07:41jeffedoceo: because i'm curious, what kinds of things are you intending to hack on?
#19:12:07jeffedoceo: went looking for an example of the db trigger method of keeping track of changes for offloading to an external discovery layer. i'll look a little more later. no luck in my quick search.
#19:12:16enhancinis there a way to remove biblio.record_entry rows and free up id's?
#19:13:00jeffenhancin: you can manually adjust the sequence in postgresql, but i'm not sure i'd recommend it.
#19:15:12jeffhttp://www.postgresql.org/docs/9.1/static/functions-sequence.html is the relevant postgresql docs. Again, I'd recommend against it, at least without further details / background.
#19:15:18jeff heads afk a few
#19:29:38edoceojeff: I have no intent, is very free form at this poing
#19:29:40edoceo*point
#19:30:28edoceoThere are other aspects of Evergreen that I'm working on / looking at - but the specifics about search/index is just for me to play with
#19:30:46edoceoI learn pretty quickly when trying to scratch-build a little thing like this - even if it's done bass-ackwards
#19:33:55youdonotexist has quit IRC
#19:40:30goood has joined #evergreen
#19:40:30goood2 has quit IRC
#19:59:21moonburn has quit IRC
#20:18:29mtcarlson has quit IRC
#20:28:44enhancin has quit IRC
#20:29:01jeffedoceo: noted! it is those "other aspects" was curious about, too :-)
#20:32:58edoceojeff: oh, that stuff is boring; I just do what some libs tell me to do
#20:33:08edoceoCan you add this to the UI? = yes
#20:33:28edoceoI did some Action Trigger stuff recently (made a Stat Cat lookup during A/T processing)
#20:33:45edoceoLooked at a billing improvement; but that was ..... yea...
#20:34:50jeffheh
#20:35:01jeffbilling is fun. any details there?
#20:36:11edoceoWell, there were some certain requirements that the billing office had; and we looked at trying to add those in; but then we kept digging deeper and deeper into subject and the project got bigger and bigger
#20:36:25edoceoSo, then it was like - we'll come back to that; first glance was mis-leading
#20:36:41jeffsorry to ask to many questions -- feel free to return fire.
#20:36:45jeffgot it.
#20:37:19edoceowhat is your SSN?
#20:37:22edoceo:D
#20:37:25jeffheh
#20:38:14edoceoI'm kinda new to Evergreen but have hacked on other projects for a time; seems pretty straight forward after cresting the wave of newness
#20:38:56jeff078-05-1120 :-)
#20:38:57edoceoLacking in some docs; but someone on the mailing list was talking about that and I volunteered a guy I work for to do it - IIRC there was some funding behind it.
#20:40:31edoceoI can't really labour-of-love the Evergreen project; already kind of do that for a few others
#20:40:32goood has quit IRC
#20:40:33goood2 has joined #evergreen
#20:58:42hopkinsju has joined #evergreen
#21:19:33pmplett has joined #evergreen
#21:31:22_bott_ has quit IRC
#21:32:20_bott_ has joined #evergreen
#21:43:30reosarevok has quit IRC
#22:22:21pmplett has quit IRC
#22:40:35goood has joined #evergreen
#22:40:35goood2 has quit IRC
#22:48:08hopkinsju has left #evergreen
#22:59:33youdonotexist has joined #evergreen
#23:06:14finnx has quit IRC
#23:06:25finnx has joined #evergreen
#23:10:33goood2 has joined #evergreen
#23:10:34goood has quit IRC
#23:40:40goood2 has quit IRC
#23:40:40goood has joined #evergreen
< Tuesday, February 7th, 2012Raw Log FileThursday, February 9th, 2012 >