Open Source Integrated Library System

Evergreen on IRC

#evergreen Logs for Wednesday, November 16th, 2011

< Tuesday, November 15th, 2011Raw Log FileThursday, November 17th, 2011 >
#TimeNickMessage
#00:45:12jeffdavis has joined #evergreen
#02:45:48edoceo has quit IRC
#05:51:30moodaepo has quit IRC
#05:51:54moodaepo has joined #evergreen
#06:26:47wlayton has joined #evergreen
#06:56:28wlayton has quit IRC
#07:21:26Callender has quit IRC
#07:21:26phasefx has quit IRC
#07:24:01Callender has joined #evergreen
#07:24:01phasefx has joined #evergreen
#08:16:27Dyrcona has joined #evergreen
#08:28:29kmlussier has joined #evergreen
#08:44:17moodaepo has quit IRC
#08:47:30Callender has quit IRC
#08:48:24moodaepo has joined #evergreen
#09:05:57Callender has joined #evergreen
#09:16:32tlilleberg has joined #evergreen
#09:22:46natschil has joined #evergreen
#09:26:05jenny has joined #evergreen
#09:27:22yboston has joined #evergreen
#09:30:54dbwells_ has joined #evergreen
#09:33:42moodaepo has quit IRC
#09:34:12dbwells has quit IRC
#09:42:22moodaepo has joined #evergreen
#09:47:12DyrconaDoes tpac honor the transcendant flag on bib sources?
#09:47:45DyrconaYes, apparently it does.
#09:48:28Dyrconatsbere: had to restart memcached.
#09:58:10Meliss has joined #evergreen
#09:59:34eeevilDyrcona: that's all in the back, yeah
#10:35:10tsbereeeevil: My make_release.sh script now attempts upgrade scripts.
#10:38:57bshumtsbere++
#10:40:56eeeviltsbere: rad, sir! 2.1.1 and 2.2-a1 !!
#10:42:20tsbereeeevil: I will push a 2.1.1 test branch to working for quick review, then do the same (after building one) for 2.2-a1. Will want someone to double-check the upgrade scripts and such to make sure I didn't screw anything up on the autogenning ;)
#10:42:56eeevilI'll check it
#10:45:48tsbere gets really lazy with the upgrade scripts and doesn't attempt the "remove dupe functions" run
#10:46:07tsbereThat is a manual process. There may be reasons they show up multiple times.
#10:46:58tsbereeeevil: http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/tsbere/test_2_1_1 is my "build rel_2_1_1" auto-committed result. Re-named. ;)
#10:56:11eeevilI'll look soon!
#11:00:16tsbereeeevil: http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/tsbere/test_2_2_alpha1
#11:03:53eeeviltsbere: not critical, but we can remove the first rewrite of asset.merge_record_assets (0639) since it's then later rewritten again (0648)
#11:04:36eeeviltsbere: re 2.1.1
#11:04:44tsbereeeevil: See my "* tsbere gets really lazy with the upgrade scripts and doesn't attempt the "remove dupe functions" run" a few min ago ;)
#11:05:27tsberemake_release even *tells* me what functions may be duped. And I was still too lazy to go de-dupe them ;)
#11:06:03eeevilheh, that's fine ... we want to keep the upgrade_log entry, though, if you do go do that
#11:07:20tsbereWell, yea. The 2.2-alpha1 upgrade script will be a PITA for removing dupes from. Lots of them ;)
#11:08:34eeevilindeed ... also, since these are going out at the same time, we'd generally remove whatever is in 2.1.0-2.1.1 from 2.1-2.2a1
#11:09:08tsbereI told 2.2a1 to base itself on rel_2_1...so it only gets things that exist in master but not rel_2_1
#11:09:33tsbere(as far as previous versions go)
#11:09:53eeevilgotcha
#11:10:25eeevilwell... that seems untrue?
#11:11:17tsbereActually...my "get the list of files", now that I look at my *notes* for it, isn't ensuring that files that differ between the two aren't grabbed. Hmm....
#11:11:18eeevil0639 is in 2.1 (it's in your 2.1.1 test branch), and in 2.2a1
#11:11:41tsbereSo the files that differ only in the :eg_version being missing are in both. Will need to re-work that logic quick.
#11:11:57eeevilyaa for test runs! :)
#11:19:07natschil has quit IRC
#11:19:26natschil has joined #evergreen
#11:24:28eeeviltsbere: also, one more upgrade script coming to cover a missed explode_array->unnest conversion
#11:24:31eeevilsoon
#11:24:37eeevilthanks to berick
#11:24:52eeevilthanks to berick for the fix, not the breakage ;)
#11:27:43tsbere replaced his logic with a "compare the sets of upgrade script *numbers* between the two branches, rather than what ones differ, and only grab the ones in the new branch that aren't in the old" block
#11:28:34eeevilperfect, sir
#11:29:12tsbereStill going to require a bit of manual tweaking for the dupes, but looks like a lot *less* show up now :D
#11:30:02tsbereeeevil: Should I tag a rel_2_2 branch while I am at it? Or are we going to run rel_2_2 out of master for a bit to reduce our "commit everything going into master *twice*" headaches?
#11:32:16gmcharltwell, is anybody expecting to do any big pushes in the next couple weeks that they expect to *not* be ready for 2.2?
#11:32:21Dyrconahmmm. tpac search seems weird to me on my dev server.
#11:32:44pmpcat has joined #evergreen
#11:33:27eeeviltsbere: IMO, beta is the time for a branch, alpha is just tags
#11:33:37tsberegmcharlt: My assumption then is "said push isn't ready for master, and perhaps shouldn't be pushed at all". AKA, should live as a topic branch.
#11:33:48Dyrcona thinks it might be his data.
#11:34:08gmcharlttsbere: agreed; I'm bring the question up just so that we're explicit about it
#11:35:53matt_carlson has joined #evergreen
#11:40:17natschil has quit IRC
#11:40:37natschil has joined #evergreen
#11:43:12tsbereAnything other than the explode_array->unnest change mentioned we want to get into 2.1.1 or 2.2-alpha1? I would be willing to argue LP bugs 880906 and 870029 myself. And for 2.2 all of my open LP bugs, I guess, but that is probably a tad much and not needed for alpha1.
#11:44:58eeevil looks
#11:46:33eeevilhrm... seems we shouldn't allow you to peer to the primary bib at all
#11:46:43eeevilthat would fix it, no?
#11:47:17tsbereThat would be another way to fix it, yes. Or, you know, do *both* just for the sake of covering *all* of our bases.
#11:48:03eeevil:)
#11:48:10tsbere hasn't checked what happens if you merge record A into record B and a copy on record B was peer bibbed to A
#11:48:29tsbereI have barely checked peer bibbing at all, really
#11:48:46eeevilactually, yeah, that's enough of a use case
#11:48:59eeevilI'll push 880906 in
#11:53:39tsbereHmmm. If we have a quick fix for 891240 that might not be a bad thing to get done either.
#11:55:03natschil has quit IRC
#11:59:00dbs has joined #evergreen
#11:59:01dbs has joined #evergreen
#11:59:51dbshey berick - your unadorned URIs branch doesn't test for ind1 = 4, ind2 = [01] - deliberate?
#12:00:28Dyrconadbs: doesn't matter, it still doesn't work.
#12:00:41dbsDyrcona: orthogonal issue
#12:01:01Dyrconadbs: yes, but one thing at a time. :)
#12:01:46dbsI guess I could shut up and forget about it
#12:01:47eeeviltsbere: no immediate fix apparent for 891240
#12:02:29tsbereeeevil: Was just a thought.
#12:03:51Dyrconadbs: I'd like to see it working as it is on my dev system before tackling your issue.
#12:07:02tsbereeeevil: Thoughts on 870029 ? (I largely disagree with the OUS options, for the record)
#12:09:52berick grabs 0651
#12:10:12berickdbs: not deliberate, just an oversight
#12:10:44eeeviltsbere: I'm not itching to push that in ASAP ... if there's community consensus that the branch is Right(tm) then let's commit to getting it in for 2.2, though
#12:11:00eeevilI'm much more itchy to get 2.1.1 and 2.2-a1 out :)
#12:11:28tsbereeeevil: Heh. And a lot of other people are itching for 2.2-anything to be out ;)
#12:13:26eeevilindeed
#12:15:28kmlussier has quit IRC
#12:15:49bshumHmm, the little left/right arrows for navigating between records from a MARC expert search breaks when you reach the end of a search results page (10 results) whilst in rdetails page level.
#12:16:07bshumWe get to the tenth record and the arrow disappears, along with the "End" link
#12:16:56eeeviltsbere: stealing berick's thunder, the unnest fix is in, and in master only
#12:17:11eeevilIMO, you're clear
#12:17:55matt_carlson has quit IRC
#12:18:01matt_carlson has joined #evergreen
#12:20:16phasefxeeevil: dbs: this may be a question for one of you. I'm looking at authority/list.js with venkman, and I'm in function displayAuthorities where it tries to grab the auth.id with a dojo.query("datafield[tag='901'] subfield[code='c']", node), except, that returns an empty array, so the auth.id is stuck at 0. How can I figure out what record I'm dealing with to check to see if it's shaped poorly?
#12:20:54tsbereeeevil: I will get right on that.....after I get back from getting my lunch. ;)
#12:21:45phasefx tries to wade through "node"
#12:22:50phasefxwould node.textContent be the whole record in marc?
#12:23:05dbsphasefx: can you grab datafield[tag='100'] subfield[code='a'] and display that?
#12:23:18dbsphasefx: oh, that might work
#12:23:40phasefx"\n 00270nz 02200073o 4500\n 1158873\n 040806|| aznznbba| un and |u\n \n Osborne, Mary Pope.\n Magic tree house series.\n Merlin mission (Sound recording)\n \n \n Magic tree house series.\n Merlin mission (Sound recording)\n \n "
#12:23:59phasefxman that looks weird to me :)
#12:24:55dbsphasefx: yeah, you're missing the tags & subfields cause they're attributes, not text nodes
#12:25:15dbsBut "Osborne, Mary Pope." is the auth record in question, methinks
#12:25:26dbssounds like the auth records don't have 901 fields?
#12:26:41phasefxhrmm.. 126 such records don't have the 901 tag
#12:26:41dbswhich was a problem that was uncovered during the last EG Conf - upgraders from 1.6 -> 2.0 didn't get 901 fields on their auth records, but of course clean loads into a 2.0 system did, so it was amazingly hard to debug until the upgrade part was recognized
#12:27:12phasefxhow do you re-ingest them or whatnot?
#12:27:58phasefxooh, one was created yesterday. This is EG 2.0.7
#12:28:16eeevilphasefx: turn on force-on-same-marc and do a trivial update. you need the maintain-901 setting on as well
#12:28:31eeevilthe latter is probably your problem, then
#12:28:31phasefxmuchos gracias
#12:29:01eeevilthose are both internal flags, of course (and may both be exposed through global flags)
#12:29:45phasefxI see force_on_same_marc.. don't see one mentioning 901
#12:29:54phasefxthere's cat.maintain_controL_numbers
#12:30:13kmlussier has joined #evergreen
#12:33:13dbsis the b_maintain_901 trigger on biblio.record_entry disabled?
#12:33:26dbsoh wait - auth :)
#12:33:47dbsis the b_maintain_901 trigger on authority.record_entry disabled?
#12:33:59phasefx checks
#12:34:20phasefxit's enabled
#12:34:44dbsmaintain_901() isn't optional, so maybe the specific record has some weirdness to it that throws off the underlying regex
#12:34:53dbswe ran into that before too
#12:34:55matt_carlson has quit IRC
#12:36:01phasefxhrmm, I'll try to force update this one and see what happens
#12:36:25dbs thinks CPAN when he hears "force update"
#12:36:59phasefx thinks dpkg
#12:37:07dbsthat too :)
#12:37:46phasefxoh, that record wasn't yesterday, it was yesterday last year :D
#12:37:59phasefxtricked by coincidence
#12:38:43dbshah
#12:38:47tsbereYou were only off by one, right? ;)
#12:38:52phasefx901 successfully created :D
#12:38:55dbsy2010 problem
#12:38:56phasefxtsbere++
#12:40:07phasefxmax(create_date) = 2011-02-22.. so I think I can just update those recs and let it simmer. thanks guys
#12:46:31Dyrconaphasefx: off by 1 error. :)
#12:52:02enhancin has joined #evergreen
#12:54:25enhancinI'm having some issues while testing some patron migration, I'm not very fluent in postgresql, but I'm hoping I can morph this insert statement to only insert unique ones. I have "INSERT INTO actor.card (usr, barcode) SELECT actor.usr.id, students.barcode FROM students INNER JOIN actor.usr ON students.username = actor.usr.usrname;" Would I be able to easily put this as update or will the script be more complicated?
#12:54:28matt_carlson has joined #evergreen
#12:56:49enhancinI was thinking UPDATE actor.card SET usr = actor.usr.id, barcode = students.barcode FROM students INNER JOIN actor.usr ON students.username = actor.usr.usrname WHERE 1 NOT IN ( SELECT 1 FROM actor.usr WHERE actor.card.usr = actor.usr.id)
#12:57:07enhancinbut I'm not sure if this is correct syntax and don't want to mess it up
#12:57:37tsbereenhancin: Add a "DISTINCT ON (students.barcode)" to the select, maybe?
#12:58:15tsbere isn't 100% sure what is being attempted
#12:58:21enhancin( SELECT 1 FROM actor.usr WHERE actor.card.usr = actor.usr.id DISTINC ON students.barcode ) ? I'm not sure what you mean I've never used distinct on...maybe I should research that -.-
#12:59:48enhancinOh, i'm migrating the patron database from a different system, sorry
#12:59:53enhancini've imported a few users and now
#13:00:03enhancinI'm testing what happens when I need to 'update' user information
#13:00:46enhancin_ has joined #evergreen
#13:00:50enhancin_(got kicked off somehow)
#13:04:53enhancin has quit IRC
#13:08:25enhancin_Got it. If anyone is wondering:
#13:08:28enhancin_INSERT INTO actor.card (usr, barcode) SELECT actor.usr.id, students.barcode FROM students INNER JOIN actor.usr ON students.username = actor.usr.usrname WHERE 1 NOT IN ( SELECT 1 FROM actor.card WHERE actor.card.usr = actor.usr.id); UPDATE actor.card SET usr = actor.usr.id, barcode = students.barcode FROM students INNER JOIN actor.usr ON students.username = actor.usr.usrname WHERE actor.card.usr = actor.usr.id;
#13:08:46enhancin_inserts new records that aren't there and then updates records that already exist
#13:09:01eeeviltsbere: if you haven't started re-cutting, I want to push one thing back from master into 2.1
#13:10:17tsbereeeevil: 2.1 is easy to re-cut. 2.2-alpha1 is a PITA for the upgrade script and dupes.
#13:10:33eeevilit's jsut 2.1
#13:10:39eeevilso, I'll do it :)
#13:10:52dbs kind of worries that the flurry of commits to prep for a release runs a bit counter to the "should be able to cut a release at any point in time" ethos
#13:10:55tsbere wonders if we should just leave the dupes in 2.2-alpha1. It is an alpha, who cares about dupes in the upgrade script ;)
#13:11:28tsberedbs: There is "we are able to" and "we are *willing* to", I think ;)
#13:11:51eeevilwait ... nevermind! already backported
#13:12:07eeevildbs: agreed, fwiw
#13:12:15dbstsbere: and granted, our pace before yesterday was pretty slow - but I suspect that's because we were manually testing each commit before merging
#13:12:32jamesrf has quit IRC
#13:12:42tsbereeeevil: Any thoughts on the 2.2-alpha1 upgrade script? Should I continue de-duping the functions where I am comfortable, or just say "whatever" and go for it?
#13:12:55dbs hopes to spend some quality time with dbwells' LDAP branch soon
#13:14:21jamesrf has joined #evergreen
#13:14:24enhancin_ has quit IRC
#13:15:00eeeviltsbere: function deduping is less important that table changes, obv, so if it's just functions then I say forget it
#13:20:47tsbere reverts his manual changes, fixes the second line, and tells the thing to get back to building stuff.
#13:22:21edoceo has joined #evergreen
#13:22:23tsbereFYI, I'm lazy and want to tell my system to upload everything at once, so I am building both first ;)
#13:22:30Dyrconahmm. Template Toolkit doesn't cache compiled templates, does it?
#13:23:56StephenGWills has joined #evergreen
#13:26:13dbsDyrcona: it can do caching, if it has been enabled. we had it disabled by default
#13:26:38dbs hopes that he's remembering correctly through a sleep-deprived haze
#13:30:07bericki believe caching is on
#13:30:48Dyrconaberick: I wonder if that is why I see no change in tpac after installing your 856 branch.
#13:31:23bericki doubt it. i've never had that kind problem w/ caching
#13:38:29tsbereHuh. Apparently you can confuse the update email stuff by creating multiple branches with one push.
#13:39:00dbsI didn't think we were going to create branches for tags any more?
#13:39:41tsbereWe tack stuff onto these things that we can't, to my knowledge, drop onto tags.
#13:39:43dbsbranches-as-tags, what are we, svn? :)
#13:41:07eeevildbs: not sure our release process is amenable to how we fiddle with versions in files ... if the git tag is enough for the (later) build tool to do the version dance, then I think we should do that
#13:42:39gmcharlteeevil: in principle the tag should provide enough information for future-build-tool
#13:42:42tater-laptop has joined #evergreen
#13:42:45matt_carlson has quit IRC
#13:43:02gmcharltunless there's something other than the version number that's ultimately needed
#13:43:41tsbere dumps all the downloads into place
#13:44:05tater-home has joined #evergreen
#13:45:46eeevilgmcharlt: that's about it ... 1) update the changelog in the repo 2) archive the tag and do the dance in the clean tree 3) i18n 4) dojo 5) tar it up
#13:46:07eeeviler, 1.5) tag it
#13:46:10tsbereI never paid a lot of attention to this part of the process. Once things are uploaded, who do I contact and how? :P
#13:46:29eeeviltsbere: you have sudo on the web server?
#13:46:35tater-laptop has quit IRC
#13:46:42eeeviloh, you mean after you've dumped them in downloads/
#13:46:44tsbereeeevil: Yes. Files are literally in place.
#13:47:34eeevil-dev, subject: RELEASE TEAM, body: what you did, request for web/wiki/etc updates
#13:47:35dbstsbere: moodaepo and bshum have been the heavy hitters on the EG web
#13:47:47eeevil(that's what I do, anyway)
#13:48:03dbseeevil: oh yeah, the release team!
#13:48:08eeevil:)
#13:48:24eeevilcsharp: how 'bout that release team ML? ;)
#13:48:26eeevil ducks
#13:48:38bshumHmm?
#13:48:55bshumOh
#13:49:16eeeviltsbere: should we request a test of someone that isn't us?
#13:49:20eeevilbefore updating the web site
#13:49:33tsbere has "Testing and Wiki/Web updates would be appreciated." in his email body
#13:49:36eeevil"upgrade script WORKSFORME" or some such
#13:49:40eeevilrighto
#13:55:01bshumQuirk with downloads page actually. There's probably not enough room on the page horizontally speaking to add the fourth branch for 2.2-alpha1. Time to say goodbye to 1.6? It probably ought to be fine on larger screens, but might get weird with 1024x768 or smaller.
#13:55:08tater-home has quit IRC
#13:58:26matt_carlson has joined #evergreen
#13:59:27tsberebshum: I would be happy with a "Here is a link to git. Learn how to check out and run master" page replacing the whole thing. ;) More seriously, I think we can stop advertising 1.6 downloads.
#13:59:32berickDyrcona: about to dissapear into a meeting.. i'll look at your sample record when i return
#13:59:37gmcharltbshum: I'd be fine with removing 1.6.x from the downloads page, but since it would be good to discourage people from trying to use alphas in production
#13:59:57gmcharltputting the alpha download in a separate area below the main download table would be my suggestion
#14:00:46jamesrf+1
#14:01:12bshumSounds good. I'll work with the others and poke at that page to see about that.
#14:01:17bshumgmcharlt++
#14:01:27tsberegmcharlt: What about those of us running production off of steadily marching forward master checkouts? ;)
#14:01:57gmcharlttsbere: you crazy people aren't exactly the target audience for the downloads page ;)
#14:02:40natschil has joined #evergreen
#14:02:40matt_carlson has quit IRC
#14:02:52gmcharltthought seriously, I *am* glad that it means that there's a strong impetus to keep master stable all the time
#14:03:22jefftopic_branches++
#14:03:28jeffstable_master++
#14:04:28Dyrcona@later tell berick that sample is just a copy and paste from the OPAC, not a very good sample really.
#14:04:28pinesol_greenDyrcona: The operation succeeded.
#14:09:25DyrconaI don't really get why any projects bother with numbered releases any more.
#14:16:27enhancin has joined #evergreen
#14:17:00enhancinI'm now having an issue with just one user logging in, when I put in their password it encrypts it wrong...I'm not sure what it's even ecrypting to get that....when I update the database with a password it encrypts it correctly.
#14:18:09enhancinI ran the password through http://www.md5decrypter.co.uk/hash-a-password.aspx and checked the logs and the password it encrypted does not match -anything- on there
#14:18:29Dyrconaenhancin: when you say "logging in" how are you logging in?
#14:18:37enhancinthrough the staff client, sorry
#14:18:46enhancinthe other users seem to work fine
#14:18:47Dyrconaenhancin: the password isn't a simple md5 hash.
#14:18:57enhancinin the db it seems to be
#14:19:35Dyrconawhen you login in with the staff client, a seed is first retrieved.
#14:19:57Dyrconathat seed is hashed as md5, then concatenated to the raw password.
#14:20:09Dyrconathe result is then hashed again as md5.
#14:20:45Dyrconathe backend (not the database) stores the initial seed for the session and then does the same steps.
#14:21:01enhancinis there anything that would cause a password mismatch to always occur for just that user?
#14:21:49Dyrconayou got locked out for too many bad password attempts? *looks in tsbere's direction, though he's not at his desk to answer*
#14:22:39enhancinhmm
#14:22:49DyrconaI think something like that was part of that anti-brute force patch, but the lockout is relatively short.
#14:23:17Dyrcona could be off base. It wouldn't be the first time.
#14:23:57Dyrconawait. I think I described that incorrectly.....
#14:24:24enhancinthe brute force patch, does it modify the db to change that or just some other setting somewhere
#14:24:52Dyrconaits a setting stored in memcached. doesn't change the password in the database.
#14:25:07Dyrconabtw, i was wrong about the combining of the hashes.....
#14:25:27Dyrconathe password is hashed and combined with the raw seed, the result is then hashed again.
#14:27:54tsbereGeneral idea: Both ends have a hash of the password(the backend only has the hash in the DB, the frontend has the password and hashes it), and the seed. The seed and password hash are combined and sent back. If both ends agree, login success. If they disagree, the backend increments a counter.
#14:28:13tsbereIf the counter reaches too high the backend claims failure even on success
#14:28:36tsbereAfter a period of time (90 seconds by default) sans password attempts the counter resets to 0
#14:29:14Dyrcona"too high" is configurable, isn't it?
#14:29:28tsbereAnd "how long"
#14:29:40enhancinit's still failing and I just went out for a cigarette before I got back on here, at least 10 minutes between my last attempt and this one
#14:30:03tsbereBad username? caps lock?
#14:30:08tsbereWrong server?
#14:30:12Dyrconabad password, even.
#14:30:17tsbereDidn't restart apache after restarting backend services?
#14:30:34enhancini did an autogen.sh and apache restart before i went out
#14:30:37tsberememcached not running?
#14:30:42enhancinmemcached is running
#14:30:50enhancinall other users work
#14:31:02enhancin[INFO:29864:oils_event.c:46:1321470001161465] Creating new event: PERM_FAILURE
#14:31:08tsbereNo staff_login permission?
#14:31:23enhancinit should...let me look in the db and compare
#14:31:29tsbereBecause PERM_FAILURE isn't "Your password failed" it is "you don't have permission to do that"
#14:31:40tsbereWhich could also mean no work OU defined
#14:31:59bshumSpeaking of work_ou
#14:32:27bshumI just noticed yesterday that folks can use any staff account with STAFF_LOGIN to get in, regardless of workstation registered.
#14:32:46bshumI tried lowering the staff_login perm to system level instead of consortium, but then nobody was able to log in.
#14:33:11bshumI didn't think it used to do that
#14:33:11tsberebshum: I think STAFF_LOGIN is checked *before* workstation is looked at.
#14:33:36bshumLike you needed the right workstation + right login with the right work_ou assigned.
#14:34:33tsbereLooks like oils_auth.c uses a hardcoded -1, which I think ends up meaning "everywhere"
#14:34:41enhancini am logging in from a different workstation
#14:37:02enhancinapparently the user permissions are 400? (in the database, i looked atht permision level, 17, and that is 400)
#14:38:40Dyrconaenhancin: permission numbers are relatively meaningless. they CAN and often DO vary by Evergreen installation.
#14:38:53Dyrconathe permission code is what you want to look at.
#14:39:19DyrconaSTAFF_LOGIN is the code I think you want.
#14:39:48enhancin400 is ADMIN_ASSET_COPY_TEMPLATE
#14:40:27enhancinthis whole thing is confusing i bet i'm just lost
#14:40:28Dyrconawhen you say "the user permissions are 400" what table and column are you getting that from?
#14:41:27enhancinIn actor.usr I see the user and the usrgroup is 17? and in the permissions schema there is a user permissions map and 17 corresponds to 400..
#14:41:53enhancinpermission.usr_perm_map
#14:42:11Dyrconawhat is the actor.usr.profile? actor.usr.usrgroup doesn't mean what you think it means.
#14:42:14enhancinoh okay
#14:42:31enhancin2, which is STAFF_LOGIN
#14:42:54Dyrconanope. look up id 2 in permission.grp_tree.
#14:43:29enhancinohhh Patrons
#14:43:48enhancinit needs to be "3"? Library Staff?
#14:44:08Dyrconathat's one way, but you should make it 4 or 5 for cataloger or circulator.
#14:44:22enhancinalright
#14:45:54Dyrconayou can also add an entry in permission.usr_grp_map to add individual users to other groups. It's how I get my staff permissions.
#14:49:39tsbereAnyone have thoughts on providing the packaged linux staff clients in the future? Would not be hard to build at release time.
#14:49:44tsbere(for 2.1+ anyway)
#14:50:16DyrconaI think I'm neutral on that one, since I always build my own.
#14:51:13enhancinin my experience it's better to build it on the machine it's running on, but you could have a script that does all of that for you
#14:51:31enhancinthat goes for any program really.
#14:52:05enhancinsome people like to have individual make properties, especially gentoo users
#14:53:08DyrconaWe don't really support gentoo, do we? ;)
#14:53:45enhancinno, gentoo users support gentoo....what I'm saying is you could offer packaged linux staff clients but you should also offer the source.
#14:54:12Dyrconaenhancin: the source is always there for anyone who wants to build it.
#14:54:56enhancingood, i think the optional package would be handy for some less-techy people
#14:55:03Dyrconawhen I build my packaged linux clients for my laptop, I also build the windows client, though I never use it, 'cause its easier than not building it.
#14:55:17Dyrconaplus, some people do use it.
#14:55:27Dyrconafor my development server anyway.
#14:55:35enhancinubuntu has become really popular with people who don't really...know how to do that stuff
#14:55:50Dyrconaand some who do. :)
#14:55:56enhancinthat's where I see it being a good thing
#14:56:02Dyrconabut I hear Minty is taking over in that space.
#14:56:39enhancinI do like Ubuntu, I've converted a couple of my friends over to it because they were running Vista on 1-2GB
#14:56:41jeffMinty Minotaur -- that's the upcoming Ubuntu release, right?
#14:56:49enhancinbut for myself I couldn't use it
#14:57:03Dyrcona actually likes Unity, though I do wish it had just a tad more options. I'd rather application icons be on the right of the screen.
#14:57:52Dyrconajeff: heh. typo on my part: http://linuxmint.com/
#14:58:56Dyrconadebian => ubuntu => mint.
#14:59:45Dyrconaif ubuntu is a zulu word meaing 'couldn't install debian,' then mint is hipster slang for 'couldn't install ubuntu'?
#15:00:18enhancinI haven't really used Linux for anything personal..I'm too much of a gamer to leave Windows for that....although I do use it on my development. I used to have an 8 node cluster of 3.0 Quad cores with 4GB DDR3 memory running on Ubuntu Cloud and that thing was a BEAST. I could compile Linux so fast. I also do a lot of testing with optimization on a gentoo virtual machine
#15:00:28edoceoHow did http://open-ils.org/documentation/evergreen-schema-1.6.0.1.html get made?
#15:00:34moodaepotsbere++ # Packaged linux clients
#15:01:08Dyrcona uses *NIX-like OSes on all of his personal computers.
#15:01:24enhancinUbuntu is a 4 click install....that's sad.
#15:01:34Dyrconatsbere: sorry to derail your poll. :p
#15:02:20tsbereWait, I had a poll?
#15:02:21tsbere:P
#15:02:49matt_carlson has joined #evergreen
#15:05:15tsbere adds linux bundling to his make_release script, just for kicks
#15:05:25jeffDyrcona: yeah, sorry... i was punning. i should have ducked to make it more obvious.
#15:05:54Dyrconajeff: I thought so. what was ubuntu's M name anyway?
#15:06:24DyrconaMinty Minx would have been cute. :)
#15:06:25edoceoMaverick Meerkat
#15:06:38Dyrconaoh that's right... maverick. didn't use it.
#15:06:48edoceotoo rouge for me
#15:06:52edoceo;/
#15:07:04Dyrconatoo rouge or too rogue? :)
#15:07:21edoceoneither, it's both!
#15:10:35enhancinmy favorite is Lucid Lynx so far
#15:10:41enhancinAlthough I do love Natty Narwhal...
#15:11:04DyrconaNatty was good. Oneiric is getting better now that they've worked some of the bugs.
#15:11:25DyrconaMy favorite is FreeBSD, or OpenBSD, but the latter is slower and more conservative. :)
#15:11:42bshumI miss Natty.
#15:12:28enhancin I was mostly just talking about the name =P not sure which ubuntu is my favorite..I think we're using Lucid on this build
#15:13:05enhancinnope i'm running Natty
#15:13:23bshumThe .10's tend to be strange and unwieldy. Or so I've been told.
#15:14:05bshumIn hindsight, I should have tried Oneiric more before upgrading my laptop. But oh well.
#15:14:20Dyrconahttp://pastebin.com/C8vrmt5n
#15:14:39bshumYuck
#15:14:44bshumThat does look nasty
#15:14:56DyrconaIs it upgrade or downgrade Perl XML-RPC to fix that?
#15:15:09bshumI want to say upgrade
#15:15:23DyrconaCPAN says I have the latest....
#15:15:31bshumBut I thought we already did that in the base makefile
#15:15:42Dyrconayep...i'll check the makefile.
#15:16:04Dyrconabtw, bshum, you posted almost the identical paste in May. :)
#15:16:13bshumOh geez
#15:16:29bshumI knew that felt familiar.
#15:16:54Dyrconait works on our production server, so maybe i'll just grab its XML-RPC lib.
#15:17:18Dyrconaheh. maybe its my client.... :)
#15:17:32Dyrconadidn't try from this workstation in production, just from the laptop.
#15:17:46Dyrconano wait... client is Python. never mind.
#15:19:37matt_carlson has quit IRC
#15:19:57matt_carlson has joined #evergreen
#15:21:21Dyrconasame version of XML::RPC on both servers, 0.9....
#15:21:35Dyrconatime to check logs.
#15:21:49Dyrcona hates checking logs.
#15:23:32Dyrconahttp://pastebin.com/x699yvse
#15:23:45Dyrconatsbere probably has the answer, and all I have to do i shout to get it. :)
#15:23:48tlilleberg has quit IRC
#15:24:08tsbere has no clue
#15:26:28Dyrconadoes work in production.
#15:29:02Dyrconawow. irc logs in august and may both have references to that very error.
#15:29:06enhancin has quit IRC
#15:29:30bshumHeh
#15:29:46bshumI know ours came up because we were testing some third party vendor's tie-in
#15:30:00bshumCome to think of it, I think we had a second one start using xml-rpc too
#15:30:03Dyrconayep ours, too.
#15:30:23Dyrconaexcept, my dev machine claims to have the same version of XML::RPC as production.
#15:30:31Dyrconamaybe I'll force it to install.
#15:38:07Dyrcona bangs his head on his desk....
#15:38:24DyrconaRPC::XML not XML::RPC..... doh!
#15:38:29bshumOops, hehe
#15:38:35Dyrconatwo different packages.
#15:39:44Dyrconafor the sake of the logs, I think I'll post the steps I took to fix it, that is if this fixes it.
#15:39:59tsberebshum: You do any testing of 2.1.1/2.2-alpha1 before forwarding that on? ;)
#15:40:33bshumtsbere: Not entirely, but it's been practice in the past to have release team communications carried over the General Mailing list vs. solely dev side only.
#15:40:44tsbereok
#15:41:13tsbere only recently got this directly involved in releases ;)
#15:42:03Dyrconanope still borken.
#15:42:57matt_carlson has quit IRC
#15:43:04matt_carlson has joined #evergreen
#15:44:36Dyrconahmm.....
#15:45:27Dyrconawhee! fixed.
#15:45:37Dyrconamore work than I needed, though.
#15:46:48fortin has joined #evergreen
#15:47:14Dyrconafor the logs;
#15:48:15Dyrconato fix this Un-convertable reference: HASH, cannot use at /usr/share/perl5/RPC/XML.pm line 164 in /var/log/apache2/error.log on Ubuntu Lucid Lynx.
#15:48:23Dyrconado the following steps.
#15:48:47akilsdonk has joined #evergreen
#15:48:59Dyrconasudo aptitude purge librpc-xml-perl
#15:49:11Dyrconaperl -MCPAN -e shell
#15:49:31Dyrconanote: there should be a sudo before that ...
#15:49:46Dyrconaforce install XML::RPC
#15:50:00Dyrconainstall RPC::XML::Function
#15:50:12Dyrconathen quit CPAN.
#15:53:39Dyrconadidn't even have to restart apache. :)
#15:53:46bshumDyrcona++
#15:54:37Dyrconabshum: we get to do it all again with Precise Pangolin next April. :)
#15:54:51matt_carlson has quit IRC
#15:55:01bshumDon't remind me :(
#15:55:11bshumThough I think I'll actually try to enjoy that.
#15:56:34yboston has quit IRC
#16:00:33Meliss has quit IRC
#16:13:31kmlussier has quit IRC
#16:18:20tsbereAnyone other than myself (and Dyrcona) think it would be worth writing up a new auditor setup that can (with hopefully minimal changes) mark the user and workstation IDs down with the changed rows?
#16:22:41gmcharltas long as it recognizes that not all row changes will have an operator and/or workstation associated with them
#16:23:11gmcharltand doesn't add much overhead
#16:23:22gmcharltbut given that, yes, I can certainly see that being useful
#16:24:32eeevilsounds something like the action tracker thingy idea berick/senator/me have been kicking around
#16:24:58dbsaction trigger tracker
#16:26:12eeevilheh
#16:26:24eeevil"last time someone did something" tracker
#16:26:42berickaka, the trapper keeper
#16:26:45berickwhy? i don't know
#16:27:03eeevilberick: I LOVE IT
#16:27:05Dyrconathe trigger trapper keeper
#16:27:25jeffberick: he's on third.
#16:27:35tsberegmcharlt: I was planning on leaving things nullable. null = we had no info.
#16:28:05Dyrconabeats having to grep through logs looking for events and authtokens. :)
#16:28:10tsbereThe *hard* part is determining where db access actually happens (I don't have a list) and whether or not I have access to user/workstation info
#16:28:19berickjeff: who is?
#16:28:36jeffberick: no, who's on first.
#16:28:58eeevilborn of the "last auth time" (definition unknown) request
#16:29:03berickah, that clears it right up
#16:30:38tsbereFor the record, the level of overhead would be "each time we go to talk to the DB on a new session and have a user/workstation, make one extra call before we start" combined with "the auditor functions make an extra call or two to get the information stored before we began". Maybe always call the before we start call so it can clear any previous session info?
#16:30:59_bott_ has quit IRC
#16:32:12_bott_ has joined #evergreen
#16:32:45tsbereDoes anyone *have* such a list of places I might need to edit handy? Doubtful, I know, but I might as well ask ;)
#16:33:06eeeviltsbere: seems like an out-of-band async-called opensrf service that acts as a data sync, and wraps itself around methods that are correctly decorated (like authoritative does) would be least invasive
#16:34:10Dyrcona gets all swoony when eeevil uses that techy talk. :p
#16:34:19eeevilso, you can foo, which was registered with the audit_me=>1 register_method flag (and how to find the auth token in the param set was defined, or assumed to be param 0)
#16:34:29tsbereeeevil: Set and retrieve functions based on the information here: http://www.postgresql.org/docs/9.1/static/plperl-global.html - Call the set function before you begin your session, call the retrieve from the auditor functions.
#16:34:53eeevil(s/you can foo/you call foo/) then, that thing runs before foo is called
#16:35:13eeevilit knows the method name and the auth token, and from that can look up the user and ws_id
#16:35:48eeeviltsbere: mmmmm ... I don't think that's the right level
#16:36:09eeevilor, it's tracking something I don't see as worthwhile
#16:36:11eeevil;)
#16:36:30tsbereI want to add the workstation and user ids to the audit tables. In general. :P
#16:36:48dbs has quit IRC
#16:37:07eeevilok, gotcha
#16:37:46eeevilyou just don't want to have to correlate the auditor with logs to point the blame arrow
#16:37:47tsbereMy method requires calling something like "evergreen.set_audit_info(user, workstation)" before you start (with nulls if you don't have either). The auditors can then call evergreen.get_audit_info or similar.
#16:38:35tsbereIf I can dump the set call into the pipeline where only the things actually accessing the DB need to know about it then the rest of the system can remain blissfully ignorant of the change.
#16:39:24eeevilpcrud-driven changes would be easy ... cstore-driven, not so much
#16:39:24tsbereAlso, good luck finding the log entry some days. :P
#16:40:21eeevilthe activity log is almost always enough for blame, IME
#16:40:55tsbereSomeone wants to know what happened 15 months ago. You only keep 12 months of disk logs...
#16:41:34eeevilmeh
#16:41:39eeevil;)
#16:41:43tsbere(all in all for us going back that far is useless either way beyond figuring out what *library* was responsible, but meh)
#16:43:47tsbereeeevil: How many things use cstore directly, instead of say through CStoreEditor?
#16:43:51DyrconaHaving people complain about something on a daily basis is a good motivator for gixing it.
#16:44:20Dyrconaheh
#16:44:44DyrconaCStoreEditor works just fine without an authtoken, just so you know.
#16:44:47eeeviltsbere: unless you're using a transaction, you can't depend on cstoreeditor, either. also, have to clean up on commit or error or rollback
#16:45:27eeevilexplicitly using a transaction, I mean
#16:46:03tsbereeeevil: the indb perl method is per db connection. Call the set whenever dealing with a new opensrf-level client?
#16:46:39tsbere doesn't think you often get multiple authtokens per set of calls to the DB
#16:50:20eeevilI'd definitely want to be able to turn something like this off
#16:53:08eeevilthe set part, I mean
#16:53:30eeeviland then the get part would be an alternate auditor implementation, I'd assume
#16:53:35eeevilnot a replacement
#16:54:11tsbereThe get part would be "dump values into two variables we then add to two nullable columns"
#16:54:43eeevil(bonus points for a function that en/disables the two different implementations)
#16:55:20eeevilcalling a plperlu function when you've chosen to never turn on the set part could very well be a big performance issue
#16:56:12eeevilimagine an update to 100k copies directly in the db
#16:56:16tsbere sees BadDebt.pm, TemplateBatchBibUpdate.pm, AppUtils.pm, Ingest.pm, and a pile of support-scripts doing direct cstore write calls, he thinks
#16:57:18eeevilIngest needs to just be deleted ... but, what do those have to do with this?
#16:57:33tsbereLooking for where I would need to worry about cstore without cstoreeditor
#16:57:34tsbere:P
#16:57:37eeevilthere are plenty of things that change rows without user intervention
#16:58:15tsbereand in those cases you would get nulls in the columns
#16:58:50eeevilI still think an "action" log would be more useful for troubleshooting. record the api_name, time, user and wsid
#16:59:34tsbereWhich would need a lot more than the api_name, time, user, and wsid to even be possibly useful. Like what they were passing into the api.
#16:59:59eeeviltsbere: sure -- that's a json blob called params :)
#17:00:35eeevilbut, neither here nor there ... testing will show if the get trigger change will be a perf issue
#17:03:04eeeviltsbere: srsly, if there's little to no perf impact, I won't stand in the way!
#17:03:27eeeviljust need to argue for the devil
#17:03:32tsbereeeevil: Need to do a test to find out how calling the get would impact things. Which means writing a get. Which I am halfway to.
#17:05:07eeevilthat should be fairly simple ... a generic one would be trivial for sure, but you probably don't want to call it twice if its avoidable
#17:05:44matt_carlson has joined #evergreen
#17:13:23tsbereeeevil: SELECT evergreen.get_audit_user(), evergreen.get_audit_ws(), evergreen.set_audit_info(x, x+1) FROM generate_series(1,10000)x; takes 331 ms via pgadmin for me. And is setting/getting for every row.
#17:15:33eeeviltsbere: that's not bad
#17:15:35eeevil:)
#17:16:11eeevildon't want an evergreen.{set/get}_global_var('foo'[,'bar']) ?
#17:17:20tsbereWell, that could be done too. But I figure *one* call to set the user and workstation would be easier on at least that front ;)
#17:19:38eeeviltsbere: what about when we want to set something else? :)
#17:19:47eeevil swings 180
#17:19:52eeevilok ... out for now
#17:19:54eeeviltsbere++
#17:22:04wlayton has joined #evergreen
#17:24:45tsbere notes that speed-wise things are comparable with generics compared to specifics on getting. Setting is faster doing two at once.
#17:31:03pmpcat has quit IRC
#17:31:52Dyrcona has quit IRC
#17:39:48StephenGWills has left #evergreen
#17:54:01jeffdavisis it just me, or is action_circulation_aging_tgr incredibly slow?
#17:54:44jenny has left #evergreen
#17:54:54jeffdavisI find it takes about 6000 ms to age a circ via the trigger on a test server, vs 6 ms if I drop the trigger and manually copy the circ into action.aged_circulation/delete without trigger
#18:02:56enhancin has joined #evergreen
#18:03:28enhancinTesting patron imports, anyone know where the students table gets created when it's just plain CREATE TABLE students? it doesn't appear to use a schema
#18:04:54enhancinah in the 'public' schema just got to that one in the list of clicking through
#18:05:22phasefxit's certainly not a stock table; never heard of it before
#18:08:50Dyrcona has joined #evergreen
#18:13:20matt_carlson has quit IRC
#18:14:07matt_carlson has joined #evergreen
#18:20:02tsberephasefx: He is doing funky migration stuff and thus creating tables for joins ;)
#18:21:11enhancinyes...we're building it for schools, each semester the list of students will change so we need to account for that...
#18:21:37jeffdavisin my experience it's a good idea to confine your custom migration tables to their own schema
#18:21:56jeffdavisymmv
#18:24:32matt_carlson has quit IRC
#18:33:58Dyrconaenhancin: how do you get your list of students?
#18:34:18enhancinCSV file
#18:34:29enhancinyeah i might have to have it put it in the actor schema
#18:34:35enhancinit's complaining so much
#18:35:02Dyrconaenhancin: i'd write something in perl that reads the csv file and not load it into the database.
#18:35:19Dyrconajust a suggestion.
#18:35:26enhancinin the documentation it's pretty straightforward...I got my first import working
#18:35:35Dyrconaok
#18:35:43enhancinit's just having issues when I'm updating information and adding more now...
#18:38:00akilsdonk has quit IRC
#18:40:19fortin has quit IRC
#19:00:58jeffdavisah, slowness of aging circs was fixed back in july
#19:01:29jeffdavisI'm all about stumbling across old, already-fixed bugs these days
#19:05:35bshumI think tspindler wrote something for loading/updating student records using SQL. We derived some content off of that and did something similar at the start of this school year for our first school system.
#19:05:56bshumHaving it as a core component would be kind of neat.
#19:07:41enhancinman...this is getting more complicated. I tried to make a table in the actor schema but now I get 'relation "actor.students" does not exist' on "SELECT actor.usr.id, actor.students.barcode FROM actor.students INNER JOIN actor.usr ON username = actor.usr.usrname"
#19:08:55enhancinwow nevermind one sec
#19:09:48enhancinI wasn't using the right db on the command line -.- i've been working too long today
#19:13:12enhancinawesome...things are working now. When I type up the documentation for this I might submit it for any other schools who want some help with it
#19:21:41jeffdavisI've also written some code for importing student records from CSV, which I shared on open-ils-dev a while back.
#19:22:54jeffdavisCore functionality for this would be cool, but in my experience the import process can be quite complicated...
#19:23:09bshumAnd unique too.
#19:23:33jeffdavisYeah. We've got two libraries doing student imports and the processes have diverged substantially already.
#19:23:37bshumTrying to adopt a uniform pattern depends greatly on whether you can convince every school to agree on a standard of using fields in the patron record. Which they never do.
#19:23:44Dyrconabshum & tsbere : think I know what our problem with lost checkins is, but I'll know for sure in a few minutes.
#19:24:03bshumDyrcona: If you fix it, you will be a god among us.
#19:24:06bshumMore than usual ;)
#19:24:36tsbere is looking into his auditor stuff, and discovered that it is transaction-immune
#19:24:55tsbereThat is, the stored user and workstation stick around even after a rollback
#19:24:59jeffdavis awaits Dyrcona's apocolocyntosis
#19:26:06bshum just read Dyrcona's note
#19:26:16bshumThat conforms to some of the cases we encounter.
#19:26:42bshumBut it's most often related to backdated checkins or situations where no fines were accrued due to a policy of 0 cents per day to max of 0.
#19:27:02bshumStill, happy thoughts for you as you poke about
#19:28:04DyrconaI think it is because the circulators don't have permissions to view another library's aous for the void lost billing.
#19:28:47Dyrconaalthough, that doesn't really explain why it fails for me.
#19:28:54tsbere has fought with the "you need to be able to see that setting to do anything with it when it should be backend stuff" issue before
#19:29:02bshumActually that sounds like what mrpeters-isl faced
#19:29:12bshumHe encountered that problem when looking at that bug
#19:29:52bshumBut it turns out that Indiana doesn't void across lib lines, so they were okay with that aspect of the problem.
#19:30:20bshumOr something like that.
#19:31:29bshumBut that does make sense.
#19:31:54Dyrconaso it looks like just setting some permissions will fix it.
#19:32:17tsbereI wouldn't be surprised if there is something else going on, though.
#19:32:37Dyrconaneither would I. since I should have the permission everywhere.
#19:32:38tsbereBecause if it were permissions it wouldn't be reproducable when logged in with everything everywhere
#19:33:01Dyrconaunless it is doing a broken check that didn't get fixed.
#19:33:09tsbereBad variable reference, maybe? Or only loading local ous and then referring to a remote one?
#19:33:23Dyrconai'm looking
#19:35:20Dyrconawell, no, that shouldn't be it.
#19:36:02Dyrconait shouldn't be checking the perms, since it is passing in an editor and not setting the auth flag on ou_ancestor_setting.
#19:36:13Dyrconathat's in AppUtils.
#19:37:30enhancinanyone good with postgresql? I'm trying to get all the addresses that aren't used (for some reason I have a ton of them). I can get it to select the right ones, but once I change = to != it just select everything...sometimes like 10 times
#19:38:04bshumenhancin: What's the query look like?
#19:38:12bshumPerhaps paste us a bit
#19:38:43enhancinSELECT id FROM actor.usr_address WHERE actor.usr.mailing_address = actor.usr_address.id;
#19:38:48enhancinbut I can't reference the usr table
#19:39:09bshumPerhaps try something like:
#19:39:30enhancinI can get to the other table like this:
#19:39:31enhancinSELECT actor.usr_address.id FROM actor.usr_address, actor.usr WHERE actor.usr.mailing_address = actor.usr_address.id;
#19:40:27Dyrconaheh, finances is spelled wrong in the ou editor: "Finanaces"
#19:40:34bshumenhancin: http://pastie.org/2875313
#19:40:41bshumThat's a very simple join between two tables.
#19:41:00bshumThe AS part aliases the table name to make things shorter for referencing
#19:41:20bshumThen you can select any specific field of the combined table elements
#19:41:37bshumLike au.id to get the usr ID, or aua.street1 for the street1 entry for a given user
#19:42:15bshumSince the term "id" is used more than once (once on each table) you'd have to qualify exactly which id you were looking for.
#19:43:53bshumenhancin: for fun reading - http://www.postgresql.org/docs/current/static/tutorial-join.html
#19:44:04bshum already learned something new reviewing the doc, heh
#19:44:10enhancinbut when I get to reverse that, I can't just change = to !=
#19:44:26enhancinin the end I need to select the ones that aren't there
#19:44:27tsbereHmmm. In theory, all updates are done during a transaction, right?
#19:44:31bshumWell, then it would give you all the mailing addresses that weren't specifically assigned to a user.
#19:44:43bshumPerhaps you have more entries than you think you do.
#19:44:45eeevilenhancin: left join
#19:45:14eeevilenhancin: or not exists
#19:45:47bshumEdited the paste, perhaps like that: http://pastie.org/2875313
#19:45:50enhancinI have 39 entries in the actor.usr_address table and when I run that same query with != instead of = I get 342 rows
#19:46:35eeevilbshum: nearly ... sec
#19:47:08bshumThat doesn't sound right. Perhaps you have a total of 381 (39 + 342) in the table. Which is what I would have expected from the potential switch of just = to != in the original query.
#19:47:23eeevilhttp://pastie.org/2875326
#19:47:37bshumOooh, interesting.
#19:47:40bshumeeevil++
#19:48:02eeevilbut, that's not quite what you need either, because there's another address field. sec
#19:48:02tsberebshum: != would become a cross product of everything *not* equal. So actor.usr count - 1 * actor.usr_address count - 1?
#19:48:37bshumtsbere: Oy, and this is why I changed majors in college.
#19:48:48bshum goes to weep in the corner
#19:48:48enhancineeevil++ I could probably figure it out now
#19:49:05enhancinsorry bshum...you were close! and thanks for trying!
#19:49:57enhancin select * from actor.usr AS au RIGHT OUTER JOIN actor.usr_address AS aua ON aua.id = au.mailing_address OR aua.id = au.billing_address WHERE au.mailing_address IS NULL OR au.billing_address IS NULL;
#19:50:05eeevilhttp://pastie.org/2875326
#19:50:24tsbereeeevil++
#19:50:24enhancinshowing off unions now, eh?
#19:50:27enhancinfirst it was right outer joins
#19:50:31tsbereFor the simple to read solution
#19:50:38eeevilenhancin: ;) ... wait unit I whip out the EXCEPT
#19:50:46enhancinoh dear
#19:50:58eeevilanyway, that should get you unused addrs
#19:51:13enhancinThanks! I think postgresql needs an UPSERT
#19:52:03eeevilwith predicate locking in now, that's possible ... some day
#19:52:07eeevilUPSERT, I mean
#19:52:24eeevilbut it would be spelled MERGE, and follow the sql standard :)
#19:53:04tsbereeeevil: Can you think of anywhere that an insert, update, or delete would happen without a transaction in progress?
#19:57:42eeeviltsbere: if it goes through cstore (and friends) it's not allowed
#19:58:23eeeviland the only other app that can write is storage, and that's pretty mature, so ... nope, I don't think so
#19:59:20enhancinthat other one isn't working , I think I get the good select from the last query I posted but that one is giving me 0 rows for some reason
#19:59:55tsbereeeevil: For fun, we can use this overall storage method to pull data *out* of a rolled back transaction. Which seems crazy as far as ideas go for me, I can't think of many reasons to use this method compared to, say, doing your query to get the data and then issuing the rollback...........maybe funky needs for an upgrade script?
#20:02:18eeeviltsbere: you lost me, sorry...
#20:03:08tsbereeeevil: Long story short, global perl storage ignores rollbacks. Or rather, transactions in general.
#20:03:36eeeviltsbere: ok, right. it's just a perl var inside the running interpreter
#20:08:37eeeviltsbere: if you want to get really crazy, look into custom variable classes and set_config() with is_local true
#20:09:32eeevilwould be basically free, just needs cooperation with pg in the config file
#20:09:55enhancinright outer joins don't just by chance work on delete's do they?
#20:10:08eeevilenhancin: they do
#20:10:21enhancinorly so I can just change SELECT * to DELETE on that statement of yours?
#20:10:29enhancinthe first one
#20:10:38enhancini modified it to select the rows I wanted because the union one wasn't working
#20:11:42enhancineeevil: it's saying sntax error near RIGHT =/
#20:11:45eeevilahh... you may need to grab just the non-NULL values for the columns on actor.usr
#20:12:10eeevilenhancin: you can't just change SELECT * to DELETE
#20:12:19enhancinyeah, i figured i couldn't
#20:12:21eeevilit works a little differently
#20:12:28eeevilsec
#20:12:55enhancinselect au.billing_address from actor.usr AS au RIGHT OUTER JOIN actor.usr_address AS aua ON aua.id = au.mailing_address OR aua.id = au.billing_address WHERE au.mailing_address IS NULL OR au.billing_address IS NULL;
#20:12:55eeevilhttp://pastie.org/2875326
#20:13:32enhancinthat might just have worked
#20:13:40tsbereeeevil: Wrong url? Looks like the previous pastie
#20:13:47enhancineeevil++
#20:13:52eeeviltsbere: shift-refresh?
#20:13:55eeeviljust edited
#20:13:58eeevilpastie--
#20:14:05enhancincaching--
#20:14:07tsbereeeevil: Ah, I didn't click. Just said "that number looks familiar"
#20:14:13eeevil:)
#20:14:15enhancinmemory--
#20:14:15Dyrconatsbere: found it!
#20:14:28Dyrconait's a stupid ou setting that we haven't set.
#20:15:29eeevil steps away
#20:16:06wlayton has quit IRC
#20:16:15Dyrconatsbere: lost items usable on checkin
#20:16:22Dyrconaset to true solves it for us.
#20:16:26Dyrconadoh!
#20:16:48tsbereYea, but is it not working with that off due to a bug? :P
#20:17:07Dyrconawhat bug?
#20:17:42Dyrconawell, in the handle_lost method it sets the status to reshelving.
#20:17:49Dyrconaprobably more to it than just that.
#20:17:58enhancinI hope documentation I write can help people, this was pretty frustrating but i've learned quite a bit
#20:18:31Dyrconai'll keep poking.
#20:19:25Dyrconamight be a bug.
#20:19:28enhancinanyways. thanks. later guys
#20:19:46Dyrconaif you look at Circulate.pm line 3304, that's where the value is checked.
#20:20:08enhancin has quit IRC
#20:20:17Dyrconaif its not set, and we're not at the owning lib, we don't change the status. we also don't do anything with the billings.
#20:20:41Dyrconai'm going to unset it and try something again.
#20:21:29Dyrconait looks like it might void the billing when it gets home, but I don't seem to recall that happening in my testing earlier this week.
#20:22:37Dyrconaalso explains why it "doesn't come off the record when they check it in."
#20:26:49Dyrconaweird: my example wouldn't checkin at the home library until i voided the billings first.
#20:27:03bshumThat is weird.
#20:30:00Dyrconathat time, it worked...... but I manually made the copy lost.
#20:31:20Dyrconaah.... this is tricky, but I think I know what is happening.
#20:31:28Dyrconaneed to play some more.
#20:33:52Dyrconanot so sure now. i wonder if it has something to do with the library that set it to lost.
#20:41:02Dyrconadamn. now its working on my dev machine no matter what I do.
#20:41:15bshumHeh
#20:41:24bshumSpooooooky
#20:41:26Dyrconai'm gonna clear memcached.
#20:43:31Dyrconathis is frustrating.
#20:46:41Dyrconaif i check them in from the patron's record, it seems to work now now matter what.
#20:46:51Dyrconaeven if I'm not at the owning library.
#20:47:15Dyrconaif I check them in on the normal checkin interface, it won't remove the bills.
#20:47:33bshumWell... that's unsettling
#20:47:35tsbereNew question is thus "what are they doing differently?"
#20:47:35Dyrconabills don't come off at the home library 'cause the copy is in transit.
#20:48:04tsbereSo the copy being in transit kills the bills coming off?
#20:48:13Dyrconatsbere: right.
#20:48:18tsbereLogic needs some re-ordering, perhaps
#20:48:20Dyrconawhen it gets home, it looks like.
#20:48:32Dyrconai'll double check.
#20:49:10Dyrconayep. item is in transit.
#20:49:43Dyrconasince the item itself isn't lost when it gets home, circ doesn't try to void any billings.
#20:49:51Dyrconaso that's a part of it.
#20:51:18Dyrconaanother part is that the code that voids the billings also sets the copy status to reshelving.
#20:51:34Dyrconai think that is an error, since the copy status is set later anyway.
#20:52:10Dyrconathat maybe why (if there is a why) the void code is skipped if lost usable on checkin is off and we're not home.
#20:53:33matt_carlson has joined #evergreen
#20:53:47eeevilDyrcona: I'd suggest conferring with berick tomorrow ... he's the nominal master of run_method() ;)
#21:00:16matt_carlson has quit IRC
#21:00:52matt_carlson has joined #evergreen
#21:26:22tecoripa has joined #evergreen
#21:26:33tecoripahello, anyone here?
#21:26:59bshumMaybe
#21:27:03bshumSorta
#21:27:06bshumNot really ;)
#21:27:18Dyrconano. no one here. go away. :)
#21:27:41Dyrconajust us bug wranglers, wrangler an odd bug.
#21:29:10Dyrconatecoripa: if you have a question, just ask.
#21:29:25tecoripasorry, turned away for a second...
#21:29:34Dyrconanp
#21:29:43tecoripacircling around *almost* have my view, controller, and datbase allhooked up
#21:30:08tecoripaso: I have a new column in the table actor.stat_cat: "required", boolean
#21:30:37Dyrconaactor.stat_cat didn't have that already?
#21:30:52tecoripaand I modified stat_cat_editor.xhtml and stat_cat_editor.js to display, edit the value of this field in the Statistical Category Editor
#21:30:58bshumIt does not.
#21:31:20Dyrconaasset.stat_cat does...... that's why I assumed actor.stat_cat did.
#21:31:40tecoripaand I updated fm_IDL.xml to include this new field in the model, and I stopped/restarted evergreen
#21:31:44tecoripaso far, so good
#21:32:29Dyrcona anticipates the "but...."
#21:32:39tecoripanow: when I try to update the value of the filed in the editor, I get this error in open-ils.storage_stderr.log:
#21:33:12tecoripa** required() isn't there. Please create me somewhere (like in actor::statcat)!
#21:33:24tecoripaactor::stat_cat, really
#21:33:35tsbereYou didn't update CDBI stuff, did ya?
#21:33:36tecoripaso I'm still missing a piece somewhere
#21:33:41tecoripano, I didn't
#21:33:53tecoripaI took a look at that config file, and it was pretty terse
#21:33:55Dyrconaautogen.sh, too, needs to be run.
#21:34:13tecoripanot anything in config.pm that clearly indicated it needed to be updated
#21:34:31tecoripaI did run autogen.sh (forgot to mention that)
#21:34:46Dyrconayeah, that's often the case, but you'll find things don't work unless you do.
#21:35:12tecoripasorry, referring to...?
#21:35:18tecoripaI should do ...?
#21:35:19Dyrconafind the entry for actor::stat_cat in config.pm and required as an "essential" field.
#21:35:26Dyrcona^add
#21:35:28tecoripaah, ok
#21:35:41tecoripaI missed that in the config.pm file
#21:35:54tecoripalet me look, make sure I was editing the correct file
#21:37:15tecoripathis isn't site_perl/OpenILS/Application/Storage/CDBI/config.pm, is it?
#21:37:18Dyrconano. there isn't an entry for actor::stat_cat in there and there wouldn't be.
#21:37:31Dyrconathat's for the config schema....
#21:37:36tecoripaah, ok. I'm looking at the wrong file, then.
#21:37:49Dyrconai told you too in email. :(
#21:37:54Dyrconato....
#21:37:58tecoripalet me read that again
#21:37:59Dyrconamust be getting tired.
#21:38:12Dyrconai told you to edit the wrong file.
#21:38:17Dyrconamy mistake.
#21:38:29Dyrconai had just updated config::bib_source in some other work.
#21:38:39tecoripano sweat. I'm glad you're around to set me straight. :)
#21:39:07Dyrconaso you restarted everything....ran autogen.sh....and still get the error.
#21:39:36tecoriparight. but I have not edited *config.pm* file yet.
#21:39:43tecoripawhich apparently I still need to do.
#21:39:56tecoripa?
#21:39:58tsbereactor.pm
#21:40:05Dyrconayeah.
#21:40:09tsbereCDBI/actor.pm should have the actor::stat_cat stuff
#21:40:32DyrconaOpen-ILS/src/perlmods/lib/OpenILS/Application/Storage/CDBI/actor.pm
#21:40:36tecoripajust a sec, let me look...
#21:40:48Dyrconawasted time typing it all out.
#21:41:06tecoripafound it. beautiful.
#21:41:17Dyrconasee. i told you the wrong file. :)
#21:41:22tecoripalet me edit that file, try it again...
#21:41:27tecoripaback in a few minutes
#21:42:13bshumtsbere++ #just finished testing 2.1.1 installs from scratch on clean Squeeze/Lucid VMs
#21:42:16bshumLooks good to me so far.
#21:42:21tecoripado I need to rerun autogen.sh after editing that file?
#21:42:28tecoripaor just restart evergreen?
#21:42:38Dyrconarestart evergreen and apache.
#21:42:45Dyrconadon't think you need to run autogen.
#21:42:55tecoripaok, got it
#21:43:11Dyrconayou only need to run autogen is you change the fieldmapper idl or your org unit hierarchy.
#21:43:31Dyrcona^is^if
#21:43:37Dyrconadefinitely getting tired.
#21:43:52tecoripaok, thanks
#21:44:43Dyrconamay be a stupid question, but you are doing a make install?
#21:45:00tecoripano, not anymore
#21:45:15Dyrconaso where did you edit the file?
#21:45:23Dyrcona@quote search armenian
#21:45:23pinesol_greenDyrcona: 1 found: #5: "<senator> the armenian regression sounds like a..."
#21:45:33Dyrcona@quote get 5
#21:45:33pinesol_greenDyrcona: Quote #5: "<senator> the armenian regression sounds like a spy novel" (added by bshum at 03:44 PM, February 22, 2011)
#21:45:34tecoripaI'm editing the pm file in the site_perl lib directly right now (after making suitable backups)
#21:46:35Dyrconaok. i don't usually do that, though sometimes I just cp the file from the git working area to the site_perl directory after making changes if its only 1 file that doesn't need make to run.
#21:47:19Dyrcona@quote add <bshum> The staff client looks pretty in Armenian
#21:47:19pinesol_greenDyrcona: Error: You must be registered to use this command. If you are already registered, you must either identify (using the identify command) or add a hostmask matching your current hostmask (using the "hostmask add" command).
#21:47:20tsbere frequently edits files in place, even perl files, when editing debug lines.....but pretty much never for the CDBI/* files
#21:47:28tecoripayeah, same here. since it's just one file, and I'm on a test machine, I just copied the file, edited it, and copied it back into place.
#21:47:55Dyrcona grumbles at pinesol
#21:48:01bshumTeehee
#21:48:11tecoripawhy not edit the CDBI files in place?
#21:49:28tsbereI just don't usually.
#21:49:37tecoripaok, fair enough.
#21:49:42Dyrconai don't usually edit anything in place.
#21:49:43tsbereRarely rapid-testing changes to them
#21:50:08Dyrcona@quote add <bshum> The staff client looks pretty in Armenian
#21:50:08pinesol_greenDyrcona: The operation succeeded. Quote #20 added.
#21:50:43Dyrconai should change my staff client to French.
#21:51:13Dyrconait would be fun telling folks to click on the "annuller" button.
#21:51:18tecoripasuccess!!
#21:51:27tecoripathank you, Drycona!
#21:51:44Dyrconayw. and tsbere++
#21:51:59tecoripaI'm documenting all this, as a case study of how to make a change, starting with zero knowledge.
#21:52:06Dyrconagood idea.
#21:52:11tecoripaand thank you, too, tsbere!
#21:52:13Dyrcona@quote get 19
#21:52:13pinesol_greenDyrcona: Quote #19: "_bott_: I wish I'd get what I wanted instead of what I asked for" (added by Dyrcona at 11:02 AM, November 15, 2011)
#21:52:24Dyrconaoops. wrong quote. :)
#21:52:34Dyrconawhich is apropos.....
#21:53:02DyrconaI think this quote sums up the problem with lost billings not clearing.
#21:53:08Dyrcona@quote get 18
#21:53:08pinesol_greenDyrcona: Quote #18: "Alan McKay: Like the ski resort of girls looking for husbands and husbands looking for girls, the situation is not as symmetrical as it might seem." (added by Dyrcona at 11:51 AM, November 10, 2011)
#21:53:22bshumDyrcona++
#21:53:25bshumIndeed
#21:54:08Dyrconaall right enough fun with pinesol_green.....i have a bug to update.
#21:54:30Dyrcona yawns.
#21:54:34Dyrconathen, to bed.
#21:54:47tecoripagoodnight, sleep well
#22:11:55jeffdeath to void.
#22:12:08jeffforgiveness all the way.
#22:16:49Dyrconaall fines to /dev/null! :)
#22:16:59bshum+1
#22:17:34jeffno, we want to see history... just not change history.
#22:17:36Dyrconabshum++
#22:17:53jeffi have branches to push
#22:18:04bshumjeff++
#22:18:12bshum:D
#22:18:13Dyrconajeff: not my folks....they want those billings obliterated.
#22:18:25Dyrconajeff++
#22:18:26matt_carlson has quit IRC
#22:18:41Dyrconamore branches to climb. :)
#22:18:45jeffwe don't do refunds, and we prefer not to have history of "this was taken in as payment for THIS billing type" change
#22:19:36Dyrconawe're a consortium. "we" whatever "we" want. "we" each member on their own..... :)
#22:19:46Dyrconaoh man....
#22:19:53Dyrconathat's it. I am going to bed, now.
#22:20:11jeffyeah. we're a district with four boards of trustee.
#22:20:16Dyrconagood night or morning, afternoon, or whatever it is wherever you are.
#22:20:21jeffDyrcona: nite!
#22:20:30Dyrcona has quit IRC
#22:46:34tecoripa has left #evergreen
#22:56:06AaronZ-PLS has quit IRC
< Tuesday, November 15th, 2011Raw Log FileThursday, November 17th, 2011 >