Open Source Integrated Library System

Evergreen on IRC

#evergreen Logs for Tuesday, January 19th, 2010

< Monday, January 18th, 2010Raw Log FileWednesday, January 20th, 2010 >
#TimeNickMessage
#00:27:53pmplett has quit IRC
#00:29:48mck9 has left #evergreen
#00:32:42dbsI'll poke at this more tomorrow; will try to reproduce on lenny / pgsql 8.3; if it dies there, too, then I start disabling triggers one by one
#00:32:49dbs has quit IRC
#01:17:21pmplett has joined #evergreen
#02:15:39pmplett has quit IRC
#02:39:07pmplett has joined #evergreen
#02:53:46eguest309 has joined #Evergreen
#03:13:08pmplett has quit IRC
#03:44:33eguest309 has left #Evergreen
#04:03:34rsinger has quit IRC
#04:54:53Dmagick has quit IRC
#05:08:08brendan_ga has quit IRC
#05:45:48eguest309 has joined #Evergreen
#06:41:32arianit has joined #evergreen
#07:09:55Dmagick_ has joined #Evergreen
#07:12:34arianit has quit IRC
#07:14:36Dmagick has joined #Evergreen
#07:42:03alxp has joined #evergreen
#07:44:44mck9 has joined #evergreen
#07:51:59gmcharlt has quit IRC
#07:59:55natschil has joined #evergreen
#08:49:48gmcharlt has joined #evergreen
#08:57:09dbs has joined #evergreen
#09:02:42Meliss has joined #Evergreen
#09:03:16bshum has joined #evergreen
#09:12:17jenny has joined #evergreen
#09:20:56dbswhile looking at adding more data to the search results (URIs and call numbers - doing a scoped "limit 3" kind of thing), I was wondering why we fire off one or more calls per bib rather than passing a list of bibs in one call
#09:21:12dbsthere must be some hard-learned lesson behind that
#09:23:17senatordbs++ ("clense" made me cringe too)
#09:24:33dbssenator: heh, it was distracting me while I was trying to focus on finding existing methods that would do what I want, so it was time for it to go :) I don't mind the alias that miker_ re-added 'cause it's for out of tree stuff that I don't have to see :)
#09:24:59dbsdbs-- # using up daily smiley quotient in one post
#09:25:24gmcharltthere's a quota? oh dear
#09:25:28senatorand if i had to guess about why one or more calls are made per bib, based on response i've received to similar questions, i'd say it might have to do with creating a faster feel in the interface (under the assumption that receiving the first couple of results quickly is better than receiving all results at once, later, to some number of users)
#09:26:00senatorbut i'm not fully informed on the matter; that might not apply in this case.
#09:26:00dbsI thought of that. but the overall latency adds up
#09:26:36dbssounds like it's worth an experiment, anyway
#09:26:43dbs stumbles on
#09:27:32dbsrebuilding the trunk database schema on lenny and hoping (sort of) that there are no backend postgresql crashes there
#09:29:12senatordbs: fwiw I did that just yesterday evening with no immediately apparent DB problems
#09:29:39dbsgood! sort of. because regressions in pgsql 8.4 would be bad.
#09:30:56senatorohhh 8.4... don't know about that
#09:31:06senatorbut i look forward to hearing how this goes
#09:33:58dbs8.4 had some weird segfaults early on, but most of them seem to have been fixed by 8.4.2
#09:34:05gdunbar has joined #evergreen
#09:34:31dbsGREAT. ingest with postgresql 8.3 / debian lenny worked (phew)
#09:35:42dbsso I guess I have two options on karmic: build postgresql 8.4.2 from scratch to see if it's a flawed package; if that still fails, build postgresql 8.3.x from scratch to confirm that it's an 8.3 vs. 8.4 thing
#10:00:32eguest309_ has joined #Evergreen
#10:01:03eguest309 has quit IRC
#10:01:04eguest309_ is now known as eguest309
#10:12:33gmcharlt has quit IRC
#10:18:37jenny has quit IRC
#10:29:42moodaepo has joined #evergreen
#10:38:22r123 has joined #evergreen
#10:40:45sfortin has joined #evergreen
#11:31:43gmcharlt has joined #evergreen
#11:41:28chrissharp123 has quit IRC
#11:41:28dbwells has quit IRC
#11:41:28jeff_ has quit IRC
#11:41:38chrissharp123 has joined #evergreen
#11:41:59dbwells has joined #Evergreen
#11:48:19dbs is surprised to find out that the buildbot is running on debian testing
#11:49:36miker_dbs: was the schema in your 8.4 db an upgraded one or fresh, when it was segfaulting?
#11:49:52dbssounds like "apt-get install python-lxml" should resolve the problem
#11:49:59dbsmiker_: it was fresh
#11:50:11miker_the most common cause of segfaults in pg backends (for me) has been mixed plperl/plperlu usage
#11:50:15miker_hrm... k
#11:50:31miker_well, everything should be plperlu now, so fresh rules that out
#11:53:26dbsI also dropped the cluster and recreated it with a C locale to hew as closely as possible to mainline, but still saw the same segfaults
#11:55:34eguest309 has quit IRC
#11:55:52dbsI could try not adding plperl to see if that makes a difference
#11:56:28eguest309 has joined #Evergreen
#11:57:41natschildoes cstore return the new object on an update?
#11:59:35jeff_ has joined #evergreen
#12:06:59berickPG people... I have, say, 10 rows in a table and I want to concatenate the values from a given column for every row in said table into a single value/string
#12:07:12berickcan that be done with sql or does that require external processing?
#12:08:47miker_natschil: it does
#12:09:29natschilmiker_: are you sure? it seems to return the id of the new object for me
#12:09:30miker_berick: SELECT array_to_string(array_accum(column_name),' ') FROM foo;
#12:09:46miker_natschil: hrm... no, you're probably right ... sorry
#12:10:07natschilmiker_: np
#12:10:10miker_yes, create returns the new object, update returns the id on success
#12:10:35berickthanks, miker_!
#12:10:49miker_I was thinking that update returned the object ... but, only create as a "quiet" setting (that does anything)
#12:11:04jenny has joined #evergreen
#12:11:25miker_berick: that's assuming you want ' ' as the joiner, and it doesn't protect against nulls in the column
#12:11:46miker_to do that, s/column_name/coalesce(column_name,'')/
#12:12:20berickI'm learnding
#12:12:22berickthanks again
#12:15:46dbsberick: alternative to moving the "DROP TRIGGER" stmt outside of the transaction is to use "DROP TRIGGER IF EXISTS..." because I think in-db ingest is targeted at pgsql 8.2 and above
#12:16:05dbs learns from miker_ as well
#12:16:35miker_heh
#12:17:02miker_I suggested the after-commit to berick, actually, since he was complaining about 8.1 ;)
#12:17:55dbsthe IF EXISTS syntax is used in reporter-schema.sql already
#12:18:09dbsin the definition of reporter.disable_materialized_simple_record_trigger
#12:18:38_dkyle_ has joined #evergreen
#12:18:56miker_I thought that got changed to use the plpgsql exception mechanism
#12:19:31dbsthat's what I'm seeing in trunk
#12:21:21dbs added it in http://svn.open-ils.org/trac/ILS/changeset/14992/trunk/Open-ILS/src/sql/Pg/reporter-schema.sql
#12:22:30dbsrel_1_6_0 has the exception approach: http://svn.open-ils.org/trac/ILS/changeset/14996/branches/rel_1_6_0/Open-ILS/src/sql/Pg/reporter-schema.sqlhttp://svn.open-ils.org/trac/ILS/changeset/14996/branches/rel_1_6_0/Open-ILS/src/sql/Pg/reporter-schema.sql
#12:22:56dbssurely we're not trying to maintain 8.1 support in trunk?
#12:24:20dbsthat comment should be s/stragglers using PostgreSQL 8.3 with 1.6/stragglers using PostgreSQL 8.1 with 1.6/
#12:57:20grantjohnson_ has joined #evergreen
#13:07:33grantjohnson_I don't get it.
#13:13:00natschil has quit IRC
#13:30:09miker_dbs: agreed, re 1.6/8.1
#13:42:43grantjohnsonphasefx: got a sec for fast-extract?
#13:44:15jenny1 has joined #evergreen
#13:45:11phasefxgrantjohnson: what's up?
#13:46:12grantjohnson:phasefx Could not create file parser context for file "SYSCONFDIR/opensrf_core.xml"
#13:48:20grantjohnsonphasefx: Don't know if this is an environ. variable, php var or I'm missing a config file.
#13:48:33phasefxgrantjohnson: probably a literal string in the source code
#13:48:59atzlooks like a make variable
#13:49:07phasefxa patch to make it smart (and default to /openils/conf/) would be great
#13:49:26grantjohnsonphasefx: doesn't exist here. No such file or directory at /usr/local/share/perl/5.8.8/OpenSRF/Utils/Config.pm line 328
#13:49:54atzdifferent perl version? 5.10 maybe?
#13:50:00grantjohnsonchecking fast-extract
#13:50:34phasefxyou'll need to run it somewhere with OpenSRF, etc. installed. May get faster extract times on your database box, but if it's a dedicated box, might not have OpenSRF on it
#13:51:35grantjohnsonphasefx: it's hard coded in fast-extract.
#13:51:36grantjohnsonmy ($config, $delim, $after,$deleted) = ('SYSCONFDIR/opensrf_core.xml', ' | ');
#13:52:08phasefxyeah, change that to /openils/conf/opensrf_core.xml and you should be good
#13:52:30grantjohnsonya... doh. Thanks
#13:52:46dbsphasefx: ./configure should fix that
#13:52:52dbserrr, grantjohnson
#13:52:58phasefxdbs: ah
#13:53:27dbsif you're copying directly from Open-ILS/examples, then you'll get the unfixed var
#13:54:57gmcharlt has quit IRC
#13:57:30dbs will check the autoconf stuff to see if it has fast-extract in its awareness
#13:58:25grantjohnsonyummy
#13:59:20dbsnope, it doesn't, or else the src file would be fast-extract.in and autoconf would generate fast-extract
#14:00:30phase_bb has joined #evergreen
#14:01:53jenny has quit IRC
#14:01:58dbs starts poking at it
#14:06:02grantjohnsonfast-extract seems to be running
#14:17:27rsinger has joined #evergreen
#14:23:48eguest309 has quit IRC
#14:26:19B_Bonner has joined #evergreen
#14:32:24rsinger has quit IRC
#14:35:35dbsokay, fix committed for fast-extract in trunk
#14:49:06mrpeters-isl has joined #evergreen
#14:49:59mrpeters-islall, is there currently any support for holding specific information such as "autographed copy", etc or would that only be available as an item level circulation alert
#14:52:05phasefxdepends on what you need that information for. You have copy level notes that don't cause alerts
#14:52:22phasefxand of course, stat cats
#14:52:37mrpeters-islphasefx: i thought so, but somehow, ive never come accross where to access copy level notes
#14:53:34phasefxthey're pretty buried (original goal was to de-emphasize notes over stat cats). In the opac, if you drill down to the item level, you'll see notes marked public. In the staff client, you have to go through the item attribute editor
#14:53:54mrpeters-islok
#14:54:01mrpeters-island where to copy stat cats appear? in opac as well?
#14:54:14phasefxyes, if public
#14:54:23mrpeters-islok, so drill down to "copy details"
#14:54:30mrpeters-island it would appear there, if public
#14:55:01mrpeters-islare those searchable?
#14:55:14phasefxno
#14:55:28mrpeters-islok, but im sure they could be made searchable for the right price
#14:55:49phasefxI'd imagine so :)
#14:56:11phasefxyou could of course embed notes in the MARC, and those could more easily be made searchable
#14:57:02mrpeters-islphasefx: that possible in a consortial setting though, where all libraries may be on the same bib, but only one of the holdings is that special copy with the special note
#14:58:38phasefxyeah, putting stuff in the MARC wouldn't scope it to specific libraries (except in the case of the MFHD stuff)
#15:00:38dbsor 856's
#15:01:09phasefxShelving Locations are searchable; you could abuse those (Autograph Shelf)
#15:02:02phasefxmrpeters-isl: if what you're wanting to cater to is vanity searches (donated by X), publishing bookbags with those books could be an option
#15:02:54grantjohnson_ has quit IRC
#15:43:09alxp has quit IRC
#15:48:39dbwellsHey folks, I think I have tracked down a bug in Vandelay and am hoping for some confirmation/help.
#15:49:15dbsconfigure.ac now has a template for checking additional default directories for expected files / libdirs (I'm looking at you, Jeroen, and your Fedora)
#15:49:22dbwellsIn the ingest_bib_items function, we have "FOR item_rule IN SELECT r.* FROM actor.org_unit_ancestors( queue_rec.owner )..."
#15:49:49dbsdbwells: I know very little about the workings of vandelay, alas - and have to head home. will peek at irc later
#15:50:29dbwellsbut queue_rec.owner is an actor.usr id, not an actor.org_unit id. This seems to explain why it works for Admin and nobody else. Am I reading it right?
#15:53:04_bott_dbs: I was just looking for something re: your 856 comment. Is that the display without a copy for electronic titles?
#15:53:56dbsdbwells: sounds like it wants a "select home_ou from actor.usr where id = queue_rec.owner" substituted there
#15:54:06dbs(off the top of my head)
#15:54:35dbs_bott_: aye - but it would be a huge stretch to use that for "autographed copies" :)
#15:54:59dbwellsdbs: I think so too. I will try to replace the function and see where that gets me. Thanks!
#15:55:45_bott_dbs: right, I'm looking for what you demo'd at the conference, but haven't located details
#15:56:59dbs_bott_: ah - basically, if your 856 has a subfield $9 with the shortname of the org_unit that you want to scope it to, ingest will generate the required asset.uri / asset.uri_call_number_map / asset.call_number entries
#15:57:39_bott_dbs: perfect, exactly what I was looking for!
#15:59:01Meliss has quit IRC
#16:04:13dbs has quit IRC
#16:22:46bshum has quit IRC
#16:32:38jenny has joined #evergreen
#16:41:37atz has quit IRC
#16:49:33jenny1 has quit IRC
#16:59:43gmcharlt has joined #evergreen
#17:02:08jeff_bott_: only works that way in 1.6, i think. let me know if you find differently. i've got some things where we want to use this soon. :)
#17:06:25moodaepo has quit IRC
#17:07:06moodaepo has joined #evergreen
#17:07:23r123 has quit IRC
#17:16:43destkopPC has joined #evergreen
#17:17:43destkopPChi there guys !
#17:18:02destkopPCi need some support
#17:19:09phasefxdestkopPC: hey, what's up?
#17:19:30destkopPCi want to setup evergreen
#17:19:34destkopPCi want to test it
#17:19:52destkopPCim writing project proposal to a local library
#17:21:10B_Bonner has quit IRC
#17:21:21atz has joined #evergreen
#17:21:27phasefxthere's a demo server running you could connect to (albeit, one with a very old version of Evergreen). There's also a more recent virtualized image you could download and run inside of either VMWare or VirtualBox. You also have the option of installing the software yourself on a linux server
#17:21:51destkopPCi want to have mine
#17:21:58destkopPCi have hosting that support perl
#17:22:00destkopPCand mysql
#17:22:22destkopPCbut i never installed perl scripts
#17:23:23destkopPCcan anyone give my hand on this ?
#17:23:40phasefxEvergreen makes use of postgresql (not mysql), apache, ejabberd, memcached, as well as perl and C. I'm betting a typical hosting company wouldn't be prepared to support such things
#17:25:30phasefxdestkopPC: you may want to start by downloading the virtual image and playing with it, just to see what you're in for. Then, if you want to build your own, you could try following the installation instructions on the wiki. Folks here and on the mailing list can help you if you get stuck, but they may not be able to guide you step by step through the entire process (we're all volunteers here, and have other jobs)
#17:27:18B_Bonner has joined #evergreen
#17:28:45PhurldestkopPC, hi
#17:28:58destkopPChi Phurl
#17:29:00destkopPC:)
#17:29:12brendan_ga has joined #evergreen
#17:30:01PhurldestkopPC, come to #flossk channel
#17:30:35jenny has left #evergreen
#17:36:39phasefx waves, heading out
#17:37:32destkopPCi c phasefx
#17:37:45destkopPCi have ubuntu server here
#17:38:17destkopPCi have a machine, i have installed ubuntu. it support postgresql as well
#17:46:22moodaepodestkopPC: You could try this > This Evergreen 1.6.0.0, Ubuntu 9.10 virtual image should run under recent versions of VirtualBox or VMWare. [ http://evergreen-ils.org/~denials/Evergreen-1600-Karmic.zip ]
#17:46:47moodaepoor did you want to install Evergreen from SCRATCH : )
#17:47:13destkopPCmoodaepo, im very very young tux
#17:47:19destkopPC:)
#17:48:23moodaepoWell then I would go with the virtual image (which I think phasefx also suggested)
#17:49:26moodaepoDownload the file I linked and while it's downloading you can install vbox using your package manager
#18:09:43brendan_ga has quit IRC
#18:09:43gdunbar has quit IRC
#18:09:43artunit has quit IRC
#18:11:02sfortin has quit IRC
#18:11:22B_Bonner has quit IRC
#18:13:22gdunbar has joined #evergreen
#18:13:24artunit has joined #evergreen
#18:45:50rsinger has joined #evergreen
#19:43:06rsinger has quit IRC
#19:52:32destkopPC has quit IRC
#20:02:50dbs has joined #evergreen
#20:05:13dbsdbwells: yeah, i realized on my walk home that the vandelay org_unit ids should probably be work_ous - looks like you came to the same conclusion
#20:25:32Meliss has joined #Evergreen
#20:34:04jeffdbs: i may have asked you before, but location uris 856 $9 stuff requires 1.6, yes?
#20:34:18dbsyep
#20:36:27Meliss has quit IRC
#20:37:10jeffthanks. :)
#21:18:54jeff digs into backdated checkins and generate_fines
#21:26:21jeffhas anyone toyed with the idea of having a user-level "max outstanding fines" that works as a cap for overdue fines?
#21:28:24phasefxI thought I heard such a desire expressed, but it may have been from you
#21:29:15jeffwe've probably raised the question before
#21:30:02jeffi'm thinking a org unit setting that generate_fines consults based on the circ user's home ou
#21:30:43jeffif defined, it consults the current outstanding bills for overdues... that might be the hard part.
#21:31:14jeffmaybe you could do circ bill balance for any circs not marked LOST...
#21:31:33jeffhard part is probably determining a logical way for it to work.
#21:34:05jeffmaybe skip the whole thing unless there's an additional argument passed to enable it.
#21:34:38jeff(so as not to look up an org unit setting every circ you generate fines for)
#21:35:15jeffthough, you should be able to cache that within the sub, so then it would just be one lookup for each unique user ou.
#21:35:43phasefxI'm wondering if it would be easier (though kludgier) to have some cron job go around and tweak circs so that they stop generating fines as needed
#21:35:51jeff(does opensrf cache that kind of thing on a lower level?)
#21:37:22jeffi'm trying to determine if i want the fines to resume if the user makes a payment but doesn't return/renew/etc the item(s).
#21:37:37phasefx doesn't know.. "I'm not even sure if memcached is an OpenSRF thing or an OpenSRF application thing. I suspect the latter, and that OpenSRF doesn't do any caching on its own
#21:37:39jeffi think i do, though that would be a little silly of the user.
#21:38:45jeffyeah, a call to get the ou setting could cache it pretty easily, you'd make the opensrf call to the server, but not hit the db. not sure how much you'd save there, if any.
#21:39:17jeffprobably a good idea to reduce calls where you can :)
#21:39:35phasefxyeah, I'd cache in whatever app I was using
#21:41:05dbsOh cool! with the XC NCIP toolkit we could use a REST interface to get XML from Evergreen... it's about time!
#21:41:21phasefxdbs: it's almost like Jangle, without the NCIP :)
#21:42:00dbsOr unAPI, or supercat, or...
#21:42:35phasefxwhat they're really saying is that they want a standard cross-ILS api that isn't NCIP
#21:45:16jeff learns about money
#21:45:16jeff:P
#21:49:24phasefxjeff: so are you guys using that payment<->billing mapper thing now for linking stuff within a transaction?
#21:59:40dbsmmm, ISSNs are getting truncated at their first 4 digits in reporter.materialized_simple_record in trunk
#22:01:45dbsprolly cause that's how they're getting shoved into metabib.real_full_rec...
#22:01:47dbs keeps digging
#22:08:54dbshmm, naco_normalize, you not being my friend?
#22:12:21dbsyeah, removing the \- from the pertinent tr/// line in public.naco_normalize avoids that issue. Not sure what ramifications that would have otherwise, though
#22:13:39miker_dbs: it deviates from the naco normalization standard :)
#22:14:21miker_how about a special case, like the controlfield stuff, to skip 022?
#22:14:36dbswas thinking about that
#22:15:00dbsmy unicode is getting corrupted, too, might need to add Encode + decode_utf8() in there
#22:15:23dbse.g. é was _not_ becoming e, but garbled
#22:15:42miker_in naco_norm?
#22:17:40dbsyeah
#22:17:56dbsbut nope, that doesn't fix the problem. Weird.
#22:17:58miker_hrm... pg should be passing the data in as utf8
#22:18:49dbsoh. hmm.
#22:19:02jeffphasefx: i don't think we need that to do what i'm thinking with fines right now, but yes... i hope to test that Real Soon Now!
#22:19:35phasefxjeff: yeah, I was just curious
#22:19:40jeff(hi miker_!)
#22:19:40dbsbiblio.record_entry.marc has "Qu&#xE9;bec" already, mayhaps naco_normalize is corrupting that?
#22:21:59miker_jeff: hey :)
#22:22:56jeffapologies to all in advance, as i will likely be full of fines-related questions over the next week or two. :-)
#22:23:45miker_dbs: I'm not in a position to debug right now, but if you don't find it then I'll look tomorrow morning
#22:24:06dbsno problem miker_
#22:24:14dbsthis is what I get for not testing before :)
#22:27:01miker_:)
#22:28:58jeffanyone here have access to syndetics api docs / developer docs?
#23:26:27dbshmm. I'm thinking that lc() is behaving mighty poorly, even with utf8 strings, because in the postgresql environment the locale is C
#23:48:11jeffso far from my reading the syndetics ac handler should be easy to teach to look at UPC
#23:48:35jeffopenlibrary and amazon can do it, just not as easily/blindly
#23:50:02jeff(1x1 images with 200 OK are evil)
#23:50:13dbsabsolutely agreed
#23:50:45jeffi'll bet openlibrary would be open to a tweak to give a clearer 404
#23:51:04jeffi think amazon already does, but i haven't played with their api since they started requiring the hashes
#23:51:37jeffsyndetics appears to just be "give us the isbn and the upc and the oclc and we'll either give you an image or give you a 1x1"
#23:51:58jeffwhich is good for "get me a cover by any means from syndetics", but bad for stacking ac handlers
#23:55:37dbsyeah, I'm sure openlibrary would be willing to do the right thing
#23:56:07jeffeven if it were an additional parameter so as not to break things that want the current behavior
#23:56:16jeffstrict404=true or something
#23:56:56jeffi think you can still give an image AND a 404
#23:57:05jeffbut that trips onError on an img, iirc.
#23:58:37jeffanyway, i'm going to try to get better docs, but i can get syndetics to look up with upc, now i just need to tweak the opac to extract and pass more keys. need to see if bibtemplate will be required / easier there.
< Monday, January 18th, 2010Raw Log FileWednesday, January 20th, 2010 >