Differences

This shows you the differences between two versions of the page.

--- scratchpad:random_magic_spells [2010/03/09 15:11]
dbs Note that restarting services is required after adding a new indexing class
+++ scratchpad:random_magic_spells [2010/07/28 11:05] (current)
phasefx Finding mis-matched depths with org units and org unit types, mck9++
@@ Line 1: / Line 1: @@
-=====Check Evergreen version via srfsh as suggested by  Jason Etheridge=====
+====== Troubleshooting ======
+=====Check Evergreen version via srfsh=====
    request open-ils.actor opensrf.open-ils.system.ils_version
 Doesn't //have// to be open-ils.actor any service will work.
+=====Check Evergreen version via http=====
+  http://acq.open-ils.org/gateway?service=open-ils.actor&method=opensrf.open-ils.system.ils_version
+Replace acq.open-ils.org with the hostname for the server to test.
 =====Unable to login through the web interface or staff client after a successful install?  Try Dan Scott's tip for resetting your password=====
@@ Line 10: / Line 17: @@
    # update actor.usr set passwd = 'open-ils' where usrname = 'admin';
    # \q
 =====Find unicode characters in a file=====
      perl -Mencoding=utf8 -MUnicode::Normalize -e 'while(<>){ $_=NFC($_);while(/([\x{0080}-\x{fffd}])/gc){ print "line $., $1 -> ".sprintf("\%0.4x",ord($1))."\n"; } }' filename.ext
-=====Transcoding a flat file from MARC8 to UTF-8=====
-     perl -MUnicode::Normalize -MEncode -MMARC::Charset=marc8_to_utf8 -e 'binmode(STDIN,":bytes");binmode(STDOUT,":utf8");while(<>){ Encode::_utf8_off($_); chomp; $_=marc8_to_utf8($_); print NFC($_)."\n"}' in-file > out-file
 =====Stored procedures to find and eliminate bad (truncated) UTF-8 characters in postgres=====
@@ Line 32: / Line 34: @@
      COMMIT;
-=====Magical command line for creating loader SQL=====
+===== Cancel Query/Report =====
-      ~/cvs/ILS/Open-ILS/src/extras/import/marc2bre.pl -k ../catflex.keys -c /opt/openils/conf/bootstrap.conf ../marc/xaa.mrc | ~/cvs/ILS/Open-ILS/src/extras/import/direct_ingest.pl -c /opt/openils/conf/bootstrap.conf -t 1 | ~/cvs/ILS/Open-ILS/src/extras/import/pg_loader.pl -c /opt/openils/conf/bootstrap.conf -or bre -or mrd -or mfr -or mtfe -or mafe -or msfe -or mkfe -or msefe -a mrd -a mfr -a mtfe -a mafe -a msfe -a mkfe -a msefe > xaa.sql
+If you need to cancel a query/report that is stuck and is locking up the queue we use pg_cancel_backend(backend_pid). This will kill the query and allow the clark process to 1) notice and 2) clean up after the canceled query.
+<code>
+select pg_cancel_backend(backend_pid);
+</code>
+The backend_pid comes from the procpid column of pg_stat_activity.
+<code>
+select procpid, now()-query_start, current_query as duration from pg_stat_activity where current_query <> '<IDLE>' order by 2;
+</code>
+This will give you the current queries, ordered by duration. Reports are easy to spot because the reporter uses MD5sum values for table aliases.
+===== Listing users with permissions by work_ou ====
+This is useful for auditing who has been granted privileges where:
+<code sql>
+SELECT distinct au.id, au.first_given_name, au.family_name, puwom.work_ou, aou.shortname, pugm.grp, pgt.name
+  FROM actor.usr au
+    INNER JOIN permission.usr_grp_map pugm ON au.id = pugm.usr
+    INNER JOIN permission.usr_work_ou_map puwom ON au.id = puwom.usr
+    INNER JOIN permission.grp_tree pgt ON pugm.grp = pgt.id
+    INNER JOIN actor.org_unit aou ON puwom.work_ou = aou.id
+  -- WHERE puwom.work_ou IN (110, 125, 126) -- uncomment this clause and adjust to limit to specific org_units
+  -- AND pgt.id IN (17, 18) -- uncomment this clause and adjust to limit to members of specific perm groups
+  ORDER BY family_name, work_ou, grp
+;
+</code>
+===== Comparing permissions between multiple permission groups =====
-=====To find the worst queries for the day, in the logger Pg logs=====
+<code sql>
+SELECT
+  (SELECT code FROM permission.perm_list WHERE id = perm) AS "Perm",
+  (SELECT 'Depth: ' || depth || ' Grantable: ' || CASE WHEN grantable THEN 'True' ELSE 'False' END
+     from permission.grp_perm_map AS b WHERE b.grp = ID_FOR_GROUP_X AND b.perm = a.perm) AS "Group X",
+  (SELECT 'Depth: ' || depth || ' Grantable: ' || CASE WHEN grantable THEN 'True' ELSE 'False' END
+     FROM permission.grp_perm_map AS b WHERE b.grp = ID_FOR_GROUP_Y AND b.perm = a.perm) AS "Group Y"
+FROM permission.grp_perm_map AS a WHERE grp IN (ID_FOR_GROUP_X,ID_FOR_GROUP_Y) ORDER BY 1;
+</code>
-    grep -Hn duration: pg.*.log|grep -v vacuum|awk '{print $16 " " $1}'| sed 's/:/ /g'|awk '{print $1 "\t" $2 ":" $3}'|sort -run|head
+===== Comparing permissions between multiple permission groups (variant: uses complete permission list) =====
-=====Generate counts for normalized query strings from activity.log=====
+<code sql>
+SELECT
+  code AS "Perm",
+  (SELECT 'Depth: ' || depth || ' Grantable: ' || CASE WHEN grantable THEN 'True' ELSE 'False' END
+     from permission.grp_perm_map AS b WHERE b.grp = ID_FOR_GROUP_X AND b.perm = a.id) AS "Group X",
+  (SELECT 'Depth: ' || depth || ' Grantable: ' || CASE WHEN grantable THEN 'True' ELSE 'False' END
+     FROM permission.grp_perm_map AS b WHERE b.grp = ID_FOR_GROUP_Y AND b.perm = a.id) AS "Group Y"
+FROM permission.perm_list as a order by 1
+</code>
+===== Finding mis-matched depths with org units and org unit types =====
-    grep multiclass activity.log |grep 'offset&quot;:0,'| PERL5LIB=/root/ perl -MJSON -pe 's/^.+{"searches":(.+),"org.+$/$1/gso;eval {$_ = (values(%{JSON->JSON2perl($_)}))[0]-&gt;{term};s/&quot;//gso;s/^\s*(.+)\s*/$1/gso;s/\s+/ /gso;$_=lc($_).&quot;\n"};$_="\n" if ($@)'|sort|uniq -c|sort -nr > query-counts.txt
+&lt;code sql>
+SELECT
+	child.ou_id  AS child_id,
+	child.depth  AS child_depth,
+	parent.ou_id AS parent_id,
+	parent.depth AS parent_depth
+FROM
+	(
+		SELECT
+			aou.id         AS ou_id,
+			aou.parent_ou  AS parent,
+			aout.depth     AS depth
+		FROM
+			actor.org_unit AS aou
+				LEFT JOIN actor.org_unit_type AS aout
+					ON ( aou.ou_type = aout.id )
+	) AS parent,
+	(
+		SELECT
+			aou.id         AS ou_id,
+			aou.parent_ou  AS parent,
+			aout.depth     AS depth
+		FROM
+			actor.org_unit AS aou
+				LEFT JOIN actor.org_unit_type AS aout
+					ON ( aou.ou_type = aout.id )
+	) AS child
+WHERE
+	child.parent = parent.ou_id
+	AND child.depth &lt;&gt; parent.depth + 1;
+&lt;/code>
+====== Utility ======
+===== Copying permissions from one group to another =====
+<code sql>
+INSERT INTO permission.grp_perm_map (grp,perm,depth)
+    SELECT ID_FOR_TARGET_GROUP, perm, depth FROM permission.grp_perm_map
+        WHERE grp = ID_FOR_SOURCE_GROUP AND perm NOT ( SELECT perm FROM permission.grp_perm_map WHERE grp = ID_FOR_TARGET_GROUP );
+</code>
+=====Transcoding a flat file from MARC8 to UTF-8=====
+     perl -MUnicode::Normalize -MEncode -MMARC::Charset=marc8_to_utf8 -e 'binmode(STDIN,":bytes");binmode(STDOUT,":utf8");while(<>){ Encode::_utf8_off($_); chomp; $_=marc8_to_utf8($_); print NFC($_)."\n"}' in-file > out-file
+=====Magical command line for creating loader SQL=====
+      ~/cvs/ILS/Open-ILS/src/extras/import/marc2bre.pl -k ../catflex.keys -c /opt/openils/conf/bootstrap.conf ../marc/xaa.mrc | ~/cvs/ILS/Open-ILS/src/extras/import/direct_ingest.pl -c /opt/openils/conf/bootstrap.conf -t 1 | ~/cvs/ILS/Open-ILS/src/extras/import/pg_loader.pl -c /opt/openils/conf/bootstrap.conf -or bre -or mrd -or mfr -or mtfe -or mafe -or msfe -or mkfe -or msefe -a mrd -a mfr -a mtfe -a mafe -a msfe -a mkfe -a msefe > xaa.sql
 =====Dan Scott asked for a web service that can turn MARC into XML ... well, here it is=====
@@ Line 80: / Line 171: @@
 <code perl>$str =~ s/\%([A-Fa-f0-9]{2})/pack('C', hex($1))/seg;</code>
+====== Bib-work ======
-=====Example of generating monthly search counts from the activity log=====
-<code>(for i in `perl -e 'print join " " => map { sprintf "\%02d" => $_ } 1 .. 31;'`; do (cd $i; echo $i; ~/stats/top_searches.sh 2>/dev/null ); done) | tee may_searches.txt</code>
-=====svn2cl command for generating the release ChangeLog=====
-<code>svn2cl --group-by-day --separate-daylogs -r 7200:HEAD svn/1.0-ILS/</code>
 =====How to generate metarecords for a newly loaded bib batch=====
@@ Line 117: / Line 200: @@
 </code>
-===== Listing users with permissions by work_ou ====
-This is useful for auditing who has been granted privileges where:
-<code sql>
-SELECT distinct au.id, au.first_given_name, au.family_name, puwom.work_ou, aou.shortname, pugm.grp, pgt.name
-  FROM actor.usr au
-    INNER JOIN permission.usr_grp_map pugm ON au.id = pugm.usr
-    INNER JOIN permission.usr_work_ou_map puwom ON au.id = puwom.usr
-    INNER JOIN permission.grp_tree pgt ON pugm.grp = pgt.id
-    INNER JOIN actor.org_unit aou ON puwom.work_ou = aou.id
-  -- WHERE puwom.work_ou IN (110, 125, 126) -- uncomment this clause and adjust to limit to specific org_units
-  -- AND pgt.id IN (17, 18) -- uncomment this clause and adjust to limit to members of specific perm groups
-  ORDER BY family_name, work_ou, grp
-;
-</code>
-===== Comparing permissions between multiple permission groups =====
-<code sql>
-SELECT
-  (SELECT code FROM permission.perm_list WHERE id = perm) AS "Perm",
-  (SELECT 'Depth: ' || depth || ' Grantable: ' || CASE WHEN grantable THEN 'True' ELSE 'False' END
-     from permission.grp_perm_map AS b WHERE b.grp = ID_FOR_GROUP_X AND b.perm = a.perm) AS "Group X",
-  (SELECT 'Depth: ' || depth || ' Grantable: ' || CASE WHEN grantable THEN 'True' ELSE 'False' END
-     FROM permission.grp_perm_map AS b WHERE b.grp = ID_FOR_GROUP_Y AND b.perm = a.perm) AS "Group Y"
-FROM permission.grp_perm_map AS a WHERE grp IN (ID_FOR_GROUP_X,ID_FOR_GROUP_Y) ORDER BY 1;
-</code>
-===== Comparing permissions between multiple permission groups (variant: uses complete permission list) =====
-<code sql>
-SELECT
-  code AS "Perm",
-  (SELECT 'Depth: ' || depth || ' Grantable: ' || CASE WHEN grantable THEN 'True' ELSE 'False' END
-     from permission.grp_perm_map AS b WHERE b.grp = ID_FOR_GROUP_X AND b.perm = a.id) AS "Group X",
-  (SELECT 'Depth: ' || depth || ' Grantable: ' || CASE WHEN grantable THEN 'True' ELSE 'False' END
-     FROM permission.grp_perm_map AS b WHERE b.grp = ID_FOR_GROUP_Y AND b.perm = a.id) AS "Group Y"
-FROM permission.perm_list as a order by 1
-</code>
-===== Copying permissions from one group to another =====
-<code sql>
-INSERT INTO permission.grp_perm_map (grp,perm,depth)
-    SELECT ID_FOR_TARGET_GROUP, perm, depth FROM permission.grp_perm_map
-        WHERE grp = ID_FOR_SOURCE_GROUP AND perm NOT ( SELECT perm FROM permission.grp_perm_map WHERE grp = ID_FOR_TARGET_GROUP );
-</code>
-===== Counting number of copies held by a library by LC call number range =====
-This is useful for getting a reasonable count of the number of copies of monographs held by a given library based on the first character of the LC call number. For Dewey you could adjust the substring length accordingly.
-<code sql>
-SELECT SUBSTRING(UPPER(acn.label), 1, 1) AS "LC class", COUNT(ac.id) AS "Count"
-  FROM asset.copy ac
-    INNER JOIN asset.call_number acn ON ac.call_number = acn.id
-    INNER JOIN actor.org_unit aou ON aou.id = acn.owning_lib
-    INNER JOIN metabib.rec_descriptor mrd ON mrd.record = acn.record
-  WHERE aou.id = 103
-    AND mrd.bib_level IN ('a','m') -- limit to monographs; use 'b','s' for serials
-    AND mrd.item_form NOT IN ('s') -- limit to non-electronic resources
---    AND acn.label NOT LIKE 'XX%' -- avoid ranges of legacy ephemeral garbage
-    AND ac.deleted IS FALSE
-    AND acn.deleted IS FALSE
-  GROUP BY SUBSTRING(UPPER(acn.label), 1, 1)
-  ORDER BY 1
-;
-</code>
-===== Counting titles that match a list of ISBNs owned by a given library =====
-A common task when considering the purchase of a bulk set of new titles is to check for titles that a given library already owns. One way to do this is to compare the ISBNs in the existing records vs. the potential set of new titles, assuming that you have been given a title list that includes ISBNs.
-  - Create a scratch table containing the list of ISBNs for the potential purchase, then copy the ISBNs into that table:<code sql>
-BEGIN;
-CREATE TABLE scratchpad.isbn_list (isbn text);
--- Replace ISBN_1, ISBN_2, etc with the list of ISBNs, one per line
-COPY scratchpad.isbn_list(isbn) FROM stdin;
-ISBN_1
-ISBN_2
-\.
-COMMIT;
-</code>
-  - Then match the ISBNs against the records for a given library. We make the assumption that if you have an undeleted call number attached to the pertinent record, that you own at least one copy of the record. The only real "trick" is that '''reporter.super_simple_record.isbn''' is an ARRAY column, so we have to match against the contents of that column using the '''ANY''' operator. Warning: this is slow; that's the price you pay for a non-normalized database schema.
-<code sql>
-SELECT COUNT(*)
-  FROM asset.call_number
-  WHERE record IN (
-    SELECT id
-      FROM reporter.super_simple_record rssr
-        INNER JOIN scratchpad.isbn_list sil
-        ON sd.isbn = ANY (rssr.isbn)
-    )
-  AND owning_lib = 103
-  AND deleted IS FALSE
-;
-</code>
 =====How to include a specific MARC field with a specific search class.=====
 For this example, how to include tag 590's with keyword searches:
@@ Line 229: / Line 216: @@
    WHERE field_class = 'keyword' AND name = '590';
 </code>
-  - Then restart the pertinent services ('''open-ils.storage''', '''open-ils.ingest''', '''open-ils.search''') or simply restart all Perl services (use the ''-l'' flag if you're forcing it to use '''localhost''' for a hostname):<code bash>
+  - Then restart the pertinent services ('''open-ils.storage''', '''open-ils.ingest''') or simply restart all Perl services (use the ''-l'' flag if you're forcing it to use '''localhost''' for a hostname):<code bash>
 osrf_ctl.sh -a restart_perl
 </code>
@@ Line 270: / Line 257: @@
     EXECUTE PROCEDURE serial.record_entry();
 </code>
-=====Flags to use for custom builds of postgres=====
-<code>--with-perl --enable-integer-datetimes</code>
 =====How to prune a tag under the hood=====
@@ Line 282: / Line 265: @@
 </code>
-===== Custom Dojo Build =====
+===== Reingesting Bib Records =====
-Creating a build for dojo in Evergreen will decrease the loading time for each page
+Export a list of biblio.record_entry.id entries to a file called ''records_to_ingest.txt''.  If you're using the ''\o'' command in psql, remember to use ''\t'' (tuples only) or you'll have to hand-edit the output file and trim the trailing and leading lines that are not data, such as ''(2516 rows)''.  Otherwise, use a command like:
-significantly, providing that there are enough modules being called to warrant a build
+<code sh>
-in the first place.
+psql --username=evergreen --host=localhost --dbname=evergreen -A -q -t --command="select id from biblio.record_entry;" --output=records_to_ingest.txt
-See dojo documentation re: The Package System and Custom Builds:
+</code>
-http://dojotoolkit.org/book/dojo-book-0-9/part-4-meta-dojo/package-system-and-custom-builds
-To start, you want to understand the construction of a dojo build.
+Review the output file, then:
-Each build is a profile, each profile has layers.  Each layer is made up of dojo modules.
-you can see openils.profile.js to illustrate this concept. Later, these layers will be called in your markup as scripts. Therefore, you can potentially organize your dojo builds to work harmoniously with your custom set up, utilizing custom 'layers' of grouped modules for each UI.
-Currently, Evergreen has a dojo directory located at ''/openils/var/web/js/dojo''.  In this dir you'll find the subdirectories; dijit, dojo, dojox, fieldmapper, openils, and opensrf.  There is also a JS file, DojoSRF.js.  You must maintain fieldmapper, openils, opensrf, and DojoSRF.js.  The dojo dirs (dijit, dojo, dojox) will need to be removed prior to the build.
+<code sh>
+for i in `cat records_to_ingest.txt`; do
+  echo ";request open-ils.ingest open-ils.ingest.full.biblio.record $i";
+done | sudo su - opensrf -c srfsh
+</code>
-Once you have a copy of the dojo source ( which you can find at: http://download.dojotoolkit.org/current-stable/ -- source files end in 'src'), you should have four dirs; dijit, dojo, dojox, and util. All of these files need to be copied into the ''/openils/var/web/js/dojo'' directory.
+====== Statistics ======
-When completed the ''/openils/var/web/js/dojo'' directory should contain:
+=====To find the worst queries for the day, in the logger Pg logs=====
-  dijit(from dojo source code)
+    grep -Hn duration: pg.*.log|grep -v vacuum|awk '{print $16 " " $1}'| sed 's/:/ /g'|awk '{print $1 "\t" $2 ":" $3}'|sort -run|head
-  dojo(from dojo source)
-  dojox(from dojo source)
-  fieldmapper
-  openils
-  opensrf
-  util(from dojo source)
-  DojoSRF.js
-After the ''/openils/var/web/js/dojo'' directory is properly set up, cp openils.profile.js (from /Open-ILS/examples) into the ''/openils/var/web/js/dojo/util/buildscripts/profiles'' directory.
+=====Generate counts for normalized query strings from activity.log=====
-To run the build:
+    grep multiclass activity.log |grep 'offset":0,'| PERL5LIB=/root/ perl -MJSON -pe 's/^.+{"searches":(.+),"org.+$/$1/gso;eval {$_ = (values(%{JSON->JSON2perl($_)}))[0]->{term};s/"//gso;s/^\s*(.+)\s*/$1/gso;s/\s+/ /gso;$_=lc($_)."\n"};$_="\n" if ($@)'|sort|uniq -c|sort -nr > query-counts.txt
-   cd /openils/var/web/js/dojo/util/buildscripts
+=====Example of generating monthly search counts from the activity log=====
-   ./build.sh profile=openils action=release version=1.3.1 (if using Dojo 1.3.1, otherwise make this the Dojo version you are currently running)
-Once you run the build, you should then have a directory in your dojo directory named 'release'.
+<code>(for i in `perl -e 'print join " " => map { sprintf "\%02d" => $_ } 1 .. 31;'`; do (cd $i; echo $i; ~/stats/top_searches.sh 2>/dev/null ); done) | tee may_searches.txt</code>
-The basic build works as such:
+===== Counting number of copies held by a library by LC call number range =====
+This is useful for getting a reasonable count of the number of copies of monographs held by a given library based on the first character of the LC call number. For Dewey you could adjust the substring length accordingly.
+<code sql>
+SELECT SUBSTRING(UPPER(acn.label), 1, 1) AS "LC class", COUNT(ac.id) AS "Count"
+  FROM asset.copy ac
+    INNER JOIN asset.call_number acn ON ac.call_number = acn.id
+    INNER JOIN actor.org_unit aou ON aou.id = acn.owning_lib
+    INNER JOIN metabib.rec_descriptor mrd ON mrd.record = acn.record
+  WHERE aou.id = 103
+    AND mrd.bib_level IN ('a','m') -- limit to monographs; use 'b','s' for serials
+    AND mrd.item_form NOT IN ('s') -- limit to non-electronic resources
+--    AND acn.label NOT LIKE 'XX%' -- avoid ranges of legacy ephemeral garbage
+    AND ac.deleted IS FALSE
+    AND acn.deleted IS FALSE
+  GROUP BY SUBSTRING(UPPER(acn.label), 1, 1)
+  ORDER BY 1
+;
+</code>
-Once you run the build script,
+===== Counting titles that match a list of ISBNs owned by a given library =====
-Dojo creates a 'release' dir which contains a new dojo directory (replete with subdirs of dijit, dojox, dojo, and util) in ''/openils/var/web/js/dojo/release''. Once the build is complete copy 'mydojo.js' (which can be found in ''/openils/var/web/js/dojo/release/dojo/dojo'') into ''/openils/var/web/js/dojo/dojo'' and rename as openils_dojo.js.
-Once the file is copied, you will need to edit ''/openils/var/web/templates/base.tt2'' after the line:
+A common task when considering the purchase of a bulk set of new titles is to check for titles that a given library already owns. One way to do this is to compare the ISBNs in the existing records vs. the potential set of new titles, assuming that you have been given a title list that includes ISBNs.
-      <script type="text/javascript" src="[% ctx.media_prefix %]/js/dojo/dojo/dojo.js"
-              djConfig="parseOnLoad: true, isDebug:true"></script>
-You will add:
+  - Create a scratch table containing the list of ISBNs for the potential purchase, then copy the ISBNs into that table:<code sql&gt;
-    <script type=&quot;text/javascript" src="[% ctx.media_prefix %]/js/dojo/dojo/openils_dojo.js&quot;
+BEGIN;
-            djConfig=&quot;parseOnLoad: true, isDebug:true&quot;></script>
+CREATE TABLE scratchpad.isbn_list (isbn text);
+-- Replace ISBN_1, ISBN_2, etc with the list of ISBNs, one per line
+COPY scratchpad.isbn_list(isbn) FROM stdin;
+ISBN_1
+ISBN_2
+\.
+COMMIT;
+</code>
+  - Then match the ISBNs against the records for a given library. We make the assumption that if you have an undeleted call number attached to the pertinent record, that you own at least one copy of the record. The only real "trick" is that '''reporter.super_simple_record.isbn''' is an ARRAY column, so we have to match against the contents of that column using the '''ANY''' operator. Warning: this is slow; that's the price you pay for a non-normalized database schema.
+&lt;code sql>;
+SELECT COUNT(*)
+  FROM asset.call_number
+  WHERE record IN (
+    SELECT id
+      FROM reporter.super_simple_record rssr
+        INNER JOIN scratchpad.isbn_list sil
+        ON sd.isbn = ANY (rssr.isbn)
+    )
+  AND owning_lib = 103
+  AND deleted IS FALSE
+;
+&lt;/code>
+====== Development ======
+=====svn2cl command for generating the release ChangeLog=====
+<code>svn2cl --group-by-day --separate-daylogs -r 7200:HEAD svn/1.0-ILS/&lt;/code>
+=====Flags to use for custom builds of postgres=====
+<code>--with-perl --enable-integer-datetimes</code>
+===== Custom Dojo Build =====
+Creating a build for dojo in Evergreen will decrease the loading time for each page significantly, providing that there are enough modules being called to warrant a build in the first place.  See Dojo documentation re: [[http://dojotoolkit.org/book/dojo-book-0-9/part-4-meta-dojo/package-system-and-custom-builds|The Package System and Custom Builds]] for details.
+To start, you want to understand the construction of a dojo build.  Each build is a profile, each profile has layers.  Each layer is made up of dojo modules.  You can see ''Open-ILS/examples/openils.profile.js'' to illustrate this concept. Later, these layers will be called in your markup as scripts. Therefore, you can potentially organize your dojo builds to work harmoniously with your custom set up, utilizing custom 'layers' of grouped modules for each UI.
+First, retrieve and extract a copy of the [[http://download.dojotoolkit.org/current-stable/|Dojo source]].  **NOTE:** You //must// use the source distribution for this, which contains ''src'' in the name, as only the source version contains the required layer building scripts!  Under the top-level extracted directory you should have four dirs; dijit, dojo, dojox, and util. Copy ''Open-ILS/examples/openils.profile.js'' into the ''util/buildscripts/profiles'' directory.
+To create the build:
+   cd util/buildscripts
+   ./build.sh profile=openils action=release version=1.3.1 # (if using Dojo 1.3.1, otherwise make this the Dojo version you are currently running)
-This will call the newly created profile layer of openils_dojo.js
+Once you run the build script you should then have a directory named ''release/'' as a peer to ''util/'', ''dojo/'', ''dijit/'' and ''dojox/'' in the Dojo source.  Copy the contents of ''release/dojo/'', optionally excluding ''release/dojo/util'', into ''/openils/var/web/js/dojo/''.
 ==== Creating a Custom Build ====
@@ Line 340: / Line 376: @@
 dojofind.pl will do this for you, with slight tweaking for the file path, if necessary.
-===== Reingesting Bib Records =====
-Export a list of biblio.record_entry.id entries to a file called ''records_to_ingest.txt''.  If you're using the ''\o'' command in psql, remember to use ''\t'' (tuples only) or you'll have to hand-edit the output file and trim the trailing and leading lines that are not data, such as ''(2516 rows)''.  Otherwise, use a command like:
+===== URL Format for "browse indexes" =====
-<code sh>
-psql --username=evergreen --host=localhost --dbname=evergreen -A -q -t --command="select id from biblio.record_entry;" --output=records_to_ingest.txt
-</code>
-Review the output file, then:
+    http://{hostname}/opac/extras/browse/{format, such as: xml (callnumber axis only), html[-full], marcxml[-full], or other unapi format}/{title|author|subject|topic|series|item-age|callnumber|authority.title|authority.author|authority.subject|authority.topic}/{org shortname, or -, ignored by authority.* axes but required positionally}/{target string}[/{page number offset, can be negative, defaults to 0}]?[&?status={number}...][&?copyLocation={number}...]
-<code sh>
-for i in `cat records_to_ingest.txt`; do
-  echo "request open-ils.ingest open-ils.ingest.full.biblio.record $i";
-done | sudo su - opensrf -c srfsh
-</code>
-===== Cancel Query/Report =====
-If you need to cancel a query/report that is stuck and is locking up the queue we use pg_cancel_backend(backend_pid). This will kill the query and allow the clark process to 1) notice and 2) clean up after the canceled query.
-<code>
-select pg_cancel_backend(backend_pid);
-</code>
-The backend_pid comes from the procpid column of pg_stat_activity.
-<code>
-select procpid, now()-query_start, current_query as duration from pg_stat_activity where current_query <> '<IDLE>' order by 2;
-</code>
-This will give you the current queries, ordered by duration. Reports are easy to spot because the reporter uses MD5sum values for table aliases.