We’re often asked how indexing of bibliographic material works in Evergreen. That’s a hard question to answer because, firstly, indexing is a hard problem to solve in practice. It’s also a hard question to answer because our solution is, well, mutable.

In Evergreen you can control what data is indexed, how it is presented to the indexer, how it is normalized for indexing — both as a raw value and as an index vector that is used internally — and the weights and ranking bonuses that any field can have when contributing to a result set.

All of that means indexing and searching are complex problems with complex solutions. So, as a start to explaining the process, I have created two pieces of documentation that I hope will help others understand what is going on under the covers, and perhaps spur discussion of how to improve and extend the Evergreen indexing and search infrastructure in the future.

To see how data is indexed, head over here on the Evergreen wiki. This covers the state of the art in trunk, what will become Evergreen 2.0 in the future.

Likewise, a discussion of searching is available here on the wiki. This is also targeted at trunk/2.0.

Please comment here, or start a thread on the open-ils-dev mailing list, if you’d like to take this discussion further. I look forward to any feedback!

In the next installment of the “behind the curtain” series I plan to highlight documentation describing what you can do with a search result for display purposes.

–miker

UPDATE: For some reason, WordPress decided that every sentence by the first in each paragraph should start with a question mark.  I’ve disabused it of that notion.  Also, it’s “peek”, not “peak” …

By Thursday evening, a patch for the vulnerability had been committed to Evergreen trunk, and by Friday evening that patch had been backported to the 1.6.0 branch. The Evergreen 1.6.0.6 security release was uploaded on Tuesday June 22, and it took until late Friday June 26 to write up the upgrade instructions, release notes, and update the downloads page for the http://evergreen-ils.org Web site.

Today, we worked out how to apply just the security fix to a running system, so that Evergreen libraries can close the vulnerability without having to apply the full release upgrade. The procedure is as follows:

1. Download the fixed file: http://svn.open-ils.org/trac/ILS/export/16749/branches/rel_1_6_0/Open-ILS/src/c-apps/oils_cstore.c
2. Copy oils_cstore.c over Open-ILS/src/c-apps/oils_cstore.c in the source directory you used to install your Evergreen system
3. Run ‘make’ to compile the updated libraries
4. Install the chrpath tool (“aptitude install chrpath”)
5. Run “chrpath -d Open-ILS/src/c-apps/.libs/oils_pcrud.so” to enable the library to link to the appropriate location
6. Copy Open-ILS/src/c-apps/.libs/oils_pcrud.so.* to /openils/lib/.
7. Restart the Evergreen C services by running ‘osrf_ctl.sh -a restart_c’

If you are running Evergreen 1.6, we recommend that you apply this security fix as soon as possible, then upgrade to the latest release (1.6.0.6) when you have an opportunity. Evergreen sites running releases prior to 1.6 are not affected by this vulnerability.

Dr. Veatch and I were talking about previous posts on the PINES circulation traffic on the day after Memorial Day and here are those impressive numbers showing the load on PINES, updated through this year’s figures.

These numbers are yet another demonstration of the increasing demand on PINES libraries—as if we needed one—and that Evergreen can handle a great deal of traffic.

The increase in the figures from 2009 to 2010 is extraordinary. The busiest day of PINES has a 20% increase in total circulations over last year’s and the busiest hour of the busiest day (11AM-12PM) shows a 30% increase.

Maximum circulations by date and interval

Thanks to Mike Rylander and Don McMorris for these numbers.

When everyone digs out a bit, we will check the per minute figure for 2007. I think most of us have suspected that number is in error. Mike has a theory about how it was miscalculated.

The checkins are reported for 2010 for the first time and that number adds a dimension to this set of statistics that was missing before. As evocative as the circs figures are they do not capture all that was going on. We will include these data next time.

Previous posts in this thread: 2007; 2008, and 2009 was posted on open-ils-general on May 27, 2009 at 10:45:12.

Bob Molyneux

What is the Superconsortium? It is varied elements of Evergreen community working in concert to develop capabilities within the software and within the community. A library that is a member of a consortium may give up a bit of freedom of action but in return enjoy increased services for its users by cooperating and sharing resources with other consortium members. In the Evergreen Superconsortium, different consortia are cooperating on developing capabilities for Evergreen. It is another arrow in the Evergreen quiver.

The Evergreen Superconsortium

In the first case of superconsortial cooperative development, two systems joined in paying for development of capabilities that were given to the community through the GNU General Public License (GPL). This case was shepherded by Equinox.

Given the changes in the state of Evergreen—for instance, there are more groups using it now—superconsortial cooperative development recently started via independent action of several consortia (Bibliomation, King County, SITKA, and PINES) that are working together to develop a Kids’ Catalog for Evergreen and contracting with FGI, a Seattle based firm. This was a signal event in the history of Evergreen.

Even though these different consortia are doing different things and in different places, they are working towards common goals and the Kids’ Catalog—and Evergreen—will be stronger for it.

We can, then, expect more software development as Evergreen’s capabilities increase through this method as well as more traditional methods. Just last week, Amy Terlaga sent out an email about two enhancements Bibliomation is seeking to have developed and asked if others in the community wanted to help. The trail has been blazed.

Where we are

The Superconsortium arises out of two necessary but not sufficient conditions: open source development and a Consortial Library System. The two separately each create tools to increase the capabilities of libraries through the software they use. Of course, we in the open source communities believe that open source development will provide robust software with a rapid development cycle. What joined the two factors to create the Superconsortium are the strength of the community’s actors and their vision of a new way of doing things. Superconsortial cooperation will lead to even more rapid and better software development and information exchange (data? techniques and practices?) than we have seen so far.

Quo Vadis Evergreen Superconsortium?

It is hard to peer into the future and see what is next—certainly more development with the newer consortia and as members of the community get to know each other better.

There are also possibilities for Superconsortial purchases, for example. These consortia have to move items around from library to library. If they used RFID tags, they could maintain the control over their items that UPS and FedEx can: these companies know where everything is. Can libraries do worse? It would seem to be an easy argument to make in these times that this is a matter of fiduciary responsibility. However, RFID tags are expensive.

At ALA last year, one of us asked a vendor how much it would cost to buy 10 million RFID tags? Well, he said, we would have to bid it but I would guess they would be around 17 cents a piece. This year, the number would be lower. Perhaps a Superconsortium of Evergreen consortia could consider such pooled purchases thus making it resemble the purchasing consortia that are so common in libraries. Of course, we do not have an entity to manage such purchases. And 10 million would not be near enough.

Bob Molyneux and Mike Rylander

I decided to make things easy on myself this time and am simply
linking to the wiki version of the newsletter:
http://evergreen-ils.org/dokuwiki/doku.php?id=newsletter:mar10

If you’d like to contribute to the next newsletter, feel free to post
to the mailing lists or contact volunteer newsletter wranglers more
directly at newsletter@evergreen-ils.org

Thanks!

Oh! I can’t resist including a picture from the Evergreen conference:

Dan Wells at the 2010 Evergreen Conference

If you’d like to weigh in on serials in Evergreen during its formative stages, please join the mailing lists and say hello!

Volume 3, Issue 1 – January/February 2010

As a reminder, we post this newsletter to the Evergreen general discussion list, development list, and the Evergreen blog. Cross-posting and forwarding are encouraged.

In This Issue

Evergreen Out and About, Evergreen Development and Documentation Update, Evergreen People, Evergreen Libraries, Evergreen Jobs, Evergreen Statistics, New Evergreen Libraries, Planet Evergreen, A Few Reminders, Newsletter Administrivia

Out and About: An Evergreen Calendar

Do you know of Evergreen events you’d like to share here? Please contact us at newsletter@evergreen-ils.org

Conferences

The 2010 Evergreen International Conference will be held April 21-23, 2010 at the Amway Grand Plaza Hotel in Grand Rapids Michigan. The Conference website contains general information, schedule, exhibitor information, sponsorship information, a link to the Grand Rapids Convention and Visitors Bureau, a link to the Amway for online reservations, and a link to the registration site. Please join us for an exciting 3 days of learning, sharing, networking, and fun in Grand Rapids! http://www.evergreen2010.org/

Equinox will be exhibiting Evergreen at PLA2010 (booth 1407) in Portland from March 23-27, and also at TLA in San Antonia from April 14-17.

Classes

Equinox

Equinox Software, Inc. is now offering training for Evergreen users. For more information or to register, send an email to training@esilibrary.com

Currently, Equinox Software is offering the following classes:

Acquisitions Preview

March 31 — 1pm-3pm EST

In this class, we will review the plans for the Acquisitions module, examine current functionality, and tour the most recent development. This class would be useful to administrators, prospective Evergreen users, and others interested in Acquisitions development.

The Evergreen OPAC

April 7 — 1pm-2pm EST

In this session, we will explore the Evergreen OPAC from the perspective of both patrons and staff members. This class would be useful to front line information services staff.

Circulation in Evergreen (Part 1)

April 12 — 1pm-2:30pm EST

In this session, we will focus on patron services in the Evergreen circulation module. This class would be of interest to front line circulation staff.

Circulation in Evergreen (Part 2)

April 27 — 11:30am-1pm EST

In this session, we will focus on item management in the Evergreen circulation module. This class would be of interest to front line circulation staff.

Booking in Evergreen

April 29 — 11:00am-12pm EST

In this session, we will focus on booking in Evergreen. We will discuss how to create and pick up reservations for bibliographic items. We will also discuss the reservation process for other item types, such as laptops and meeting rooms.

Lyrasis

Evergreen Cataloging Module (Live Online)

3/3/2010-3/4/2010 2:00pm-4:00pm EST

Evergreen Administration and Reports Module (Live Online)

3/10/2010, 10:00am-12:00pm EST

LYRASIS (created from a merger of SOLINET, PALINET and NELINET) has taught dozens of Evergreen classes. Lyrasis is dedicated to training and instructing Evergreen, and they welcome your comments and suggestions for courses. All of their current course offerings are continuously updated, and Lyrasis plans to add more courses in the future. For comments or questions, contact Lyrasis instructors Jennifer.Bielewski@lyrasis.org or Jenny.Liberatore@lyrasis.org

In the past, so how did it go?

* February 22, 2010 – Evergreen was the focus of an afternoon pre-conference for CODE4LIB 2010.

* February 24, 2010 – Customizing and Extending Evergreen: a guide for geeks is a half-day workshop that Dan Scott lead at the Ontario Library Association Super Conference.

Evergreen Development and Documentation Update

New Releases

A new bugfix release of Evergreen, version 1.6.0.2, was released on February 18th. It includes many fixes and updated translations, including new translations for English (UK), Spanish, and Brazilian Portuguese. For a complete list of the new fixes and features, see http://evergreen-ils.org/dokuwiki/doku.php?id=feature_list_1_6_0

On February 8th, a new version of OpenSRF was released, version 1.2.2, which includes important fixes for high traffic environments. If you are running Evergreen 1.6+ you are strongly encouraged to upgrade!

A peek in the Trunk

This is a look at some of the current differences between trunk and the 1.6 branch. It’s not comprehensive. Most of these features need testing, feedback, and/or polish, but may one day get backported to 1.6 or show up in the next major branch release.

Cataloging

Bucket Notes

Toward allowing arbitrary notes for buckets and bookbags.

Copy Location Order

Allows you to define display positions for copy/shelving locations. No longer has to be in alphabetical order.

Mint Condition flag for Holds and Copies

Optionally allows us to give a “good quality” or “mint condition” or “pristine” or “complete with all pieces” type designation to items, and allows holds to optionally be filled only by items in that condition.

Update leader in MARC record when deleting or undeleting the record

This sets the leader/05 appropriately.

Circ

Auto Checkout-attempt into renewal

If an item is already checked out to a user and the circulation is past a configured auto-renewal interval, attempt to treat the transaction as a renewal instead. Option for doing this with offline transactions as well.

Backdating

Action for backdating checkins that have already happened (e.g. checkin the item, notice that you should have backdated, and then do so after the fact)

“Scan time” field shows the true time that an item was checked in, regardless of backdating. We also record the specific workstation for all checkins.

Billable Transaction Summary with Billing Location

An augmented view of billable transaction summaries. Toward filtering by location in the billing interface.

Cap Max Fine at Item Price

Org-unit setting for capping fines at the item price. Existing Percent of Item Price functionality for fine caps trumps this setting.

Circ Counts By Year

Reportable abstraction of this data.

Circulation “Chain” Summaries

In Evergreen, renewals show up as circulations, though you’re able to select or filter for these based on renewal flags on the circs. Circulation chains allow us to group together a logical sequence of original checkout and subsequent renewals as one entity.

Circulation permit check based on claims returned threshold

Allows you to set a number for how many claims returned items are allowed on a patron’s account before requiring override for new checkouts.

Circulation receipts

Can now include patron money summary information (amount owed, etc.) in checkout receipts.

Credit Card Payments

We’re able to interface with Paypal for taking credit card payments.

Custom copy status for returned items

Allows you to designate a copy status for post-claims returned items.

Fine generation

An Evergreen installation normally has a periodic process for generating fines. To supplement that, we now generate fines for a given circulation at the time of checkin, to handle certain boundary conditions.

Floating Collections

Basic support for floating collections where items can stay where they land.

History Buckets

For supporting clearable patron and/or staff facing record of checkouts.

Hold Request Cancel/Un-Cancel

Interface toggle and related org unit settings for showing and manipulating canceled holds.

Hold Request Notes

Free-text staff notes that can optionally be printed on hold slips.

Hold retargeting trigged by certain copy updates

Checks to see if an item status change warrants a retargeting of related hold requests.

Negative Balance list

Dedicated interface for listing patrons with negative balances.

Offline username

Org-unit setting for treating patron barcodes in offline transactions as usernames based on barcode format.

Patron Claims

Can mark a circulation as Claims Never Checked Out, which changes the status to Missing.

Org-unit setting for changing the copy status to something arbitrary for Claims Returned items.

Patron retrieval by internal ID

Config setting which allows retrieval of patrons by their internal DB ID.

Pending Patrons

Staging area for patron data from self-registration, migrations, external systems, etc.

Pre-cat improvements

Circ modifier and ISBN field for pre-cataloged items. The circ modifier will carry over when/if the item is cataloged. And of course, you can create circulation rules based on an item being a pre-cat with consideration of its circ modifier.

Org-unit setting for setting the circ lib of a pre-cat. For example, you could checkout a pre-cat item at BR1, but have it’s circ lib be BR2, so that upon checkin, it’ll transit to BR2.

Top of Queue flag for Hold Requests

Brings a hold to the front of the line, or at least next to other Top of Queue holds.

Misc

Events interfaces

View (and in some cases Edit or Cancel) triggered events for patrons and copies. These can include notices, etc.

Setting Types

Better support for defining/categorizing different user and org unit settings.

Ubuntu Karmic

Pre-requisite installer target for Ubuntu’s Karmic Koala distribution.

Evergreen People

Dan Scott, Systems Librarian at Laurentian University, will be flying down to Connecticut in February to teach Bibliomation staff how to write postgreSQL queries for the Evergreen system. Dan is tailoring his lesson plan to Bibliomation’s specific reporting needs. Dan will be contributing his course materials to the Evergreen community. Class dates – February 18th and 19th.

Bibliomation, King County Library System, the SITKA Libraries (British Columbia), and the PINES Library System (GPLS) have partnered with the Seattle-based web design firm, FGI, to develop some functional specifications and a project plan for an Evergreen children’s catalog. This catalog will have graphical images to guide children to the appropriate reading material. FGI is the same company that KCLS is currently using to design the new interface for the Evergreen adult web catalog. For more information, you can email Amy Terlaga at Bibliomation (terlaga@biblio.org).

Evergreen Libraries

The Ontario Library Association (OLA) awarded two of its 2010 OLA and OLA Divisional awards to Project Conifer:
– The Ontario College and University Library Association (OCULA) Special Achievement Award
– The Ontario Library Information Technology Association (OLITA) Award for Technical Innovation

These rewards recognize not only on the partner libraries that are part of Project Conifer, but the entire Evergreen community without which Project Conifer could never have happened.

Evergreen Jobs

Equinox is currently looking for a Project Manager.

Do you know of Evergreen related jobs that you’d like to share here? Let us know at newsletter@evergreen-ils.org

Evergreen Statistics

By Bob Molyneux

Evergreen has grown rapidly and I have been keeping track as best I can by maintaining a list of the libraries running Evergreen that I can identify and, where possible, integrating that list with published national-level or provincial data.

The first table (New Libraries by Year) summarizes this growth. Bear in mind that there are uncertainties about some details and most of the published data are for 2007 so “outlets” (central library + branches + bookmobiles) may have changed but the count of systems is current.

We can see here that in 2006, the first 46 (including the State Library) systems in PINES migrated to Evergreen. There were 248 outlets by my count—not org units as Evergreen users usually report.

2007 was a slow year—the calm before the storm—with 4 new public library systems using Evergreen. This year also saw the first non-PINES libraries go live in British Columbia in what is now called SITKA.

In 2008, the pace quickend with 39 systems and 89 outlets. The first academics went live with Evergreen this year with the University of PEI.

2009 was crazy. 98 systems and 199 outlets ran Evergreen for the first time. Conifer also went live so the number of academics also increased. Conifer also has a number of academic special libraries such as health and law libraries.

The total at the end of the 2009 was 187 systems and 544 outlets using Evergreen.

Most of these are still public libraries. The second table (New Public Libraries by Year) has some summary data from these public libraries. Public library data are about the best we have; academic data are fragmentary. There are two numbers that public librarians will cite: population service and total annual circulations. What I have done here is to total these numbers from the latest figures (mostly 2007) for the libraries using Evergreen. 2007, as you can see had those four small systems go live. As I have pointed out numerous times, the story of Evergreen is a story of small public libraries but in a scalable environment. Each year since then, more libraries, more types of libraries, and bigger entities have chosen Evergreen.

Wait until this year: you ain’t seen nothin’ yet.

New Evergreen Libraries: Welcome Aboard!

* Natural Resources Canada Library completed its second and final phase of their migration on January 15, 2010. Their 13 locations across Canada are now all running Evergreen.

* SC LENDS in South Carolina welcomed three new libraries to their Evergreen consortium in January 2010: Anderson County Library, Fairfield County Library, and Florence County Library.

* The Indiana Open Source ILS Initiative has already welcomed five new libraries to their Evergreen consortium since the start of 2010: Greensburg-Decatur County Contractual Public Library, Kirklin Public Library, Ligonier Public Library, Linden-Carnegie Public Library, and Roanoke Public Library.

* Kirtland Community College in Roscommon, MI went live with Evergreen in January 2010.

* SITKA in British Columbia welcomed the Mackenzie Public Library into its consortium on February 25, 2010.

Planet Evergreen

Can’t get enough news about Evergreen open source software? Subscribe to or read Planet Evergreen, an aggregator for Evergreen-related posts. Have a blog that talks about Evergreen? To add your blog to the Planet Evergreen blog aggregator, send email to Dan Scott at dan@coffeecode.net

A Few Reminders

Evergreen has a Flickr set and a Facebook group.

Newsletter Administrivia

Feel free to forward, share, etc.! Jason sits at the top of the blame map, but we have direct edits from:

Amy Terlaga, Bibliomation, Inc., terlaga@biblio.org

Dan Scott, Laurentian University, dan@coffeecode.net

Jason Etheridge, Equinox Software Inc., jason@esilibrary.com

Mike Rylander, Equinox Software Inc.

Warren Layton, NRCan Library / Bibliothèque RNCan

And special thanks to Sally Fortin and Bob Molyneux from Equinox Software Inc. for contributions.

You can reach volunteer newsletter wranglers at newsletter@evergreen-ils.org

Licensing

This newsletter is distributed under the Creative Commons Attribution Share-Alike license, which is an open “copy-left” license similar to that used by Evergreen. If you contribute content that is copyrighted or copyrightable, please let us know if you do not agree to have it released under this license. Thanks!

Evergreen 1.6.0.2 is both a bug-fix and minor feature enhancement release: see http://evergreen-ils.org/dokuwiki/doku.php?id=feature_list_1_6_0 for an overview of the changes since 1.6.0.1. This release continues the momentum of increased community contributions of bug reports, fixes, and translations.

We invite you to try this releases, our best yet! And if you happen to find any bugs, please report them to https://bugs.launchpad.net/evergreen or to the Evergreen Development Discussion list.

As before, here is a drop-in replacement for OpenILS::Application::Search::Biblio.  We’re still planning the 1.6.0.2 release ASAP, but this addresses the known critical search issue.

As before, please let us know of any issues you encounter via either IRC or the mailing lists.

In 1.6.0.1 we fixed a bug whereby searches containing a colon, but not one that denotes a search class or modifier (like “keyword:” or “site:”), we being completely ignored.  However, that cause complications for some other searches.  The main place we’ve seen this show up is in the Z39.50 server, where requesting holdings output always causes the construction of an offending search.

This was identified by Dan Scott last night, after reports from users in the field, and I committed a fix to all open branches as of 11:00 AM EST today.  We’ll be cutting a new release, 1.6.0.2, as soon as a translation-related fix is applied but in the mean time you can see the change and download the updated file you need right here.  If you’ve installed Evergreen into the default location, drop this file into /openils/lib/perl5/OpenILS/Application/Search/ and restart the whole shebang.  Your searches will then all be happy.

If you have any questions or experience any issues, please join us in IRC or let us know on the mailing lists.

Evergreen 1.6.0.1 is both a bug-fix and translation release: see the release notes for an overview of the changes since 1.6.0.0. This release continues the healthy momentum of increased community contributions of bug reports, fixes, and new and improved translations – many thanks to all of you for making Evergreen a better system for all of us!

OpenSRF 1.2.2 (change log) follows just one week after the quietly released OpenSRF 1.2.1 (change log). Both releases focus on bug fixes and enhanced portability; most importantly for OpenSRF, it is now compatible with current versions of ejabberd, process and resource handling has been improved, and the OpenSRF HTTP translator interface delivers better browser compatibility and closer compliance to the OpenSRF-over-HTTP specification.

We invite you to try these releases, our best yet! And if you happen to find any bugs, please report them to the bug tracker for Evergreen and OpenSRF, or to the Evergreen Development Discussion list. We also welcome patches for enhancements or new features and new or updated translations.