Sunday, May 04, 2008

Installation and Configuration of LocalSolr.

A client of mine asked me to investigate localsolr which= extends Apache Solr server with the ability to perform text searches filtered on geographical distance from a given point specified by latitude and longitude. Here are the steps that I followed:

Download the Software

  1. Downland Ant from http://apache.mirror99.com/ant/binaries/apache-ant-1.7.0-bin.tar.gz
  2. Download Tomcat from http://mirror.nyi.net/apache/tomcat/tomcat-6/v6.0.16/bin/apache-tomcat-6.0.16.tar.gz
  3. Download LocalLucene from http://downloads.sourceforge.net/locallucene/locallucene-r1.5.tar.gz
  4. Download LocalSolr from http://downloads.sourceforge.net/locallucene/localsolr-r1.5.tar.gz
  5. Download Solr-1.3-dev, via svn, to get the source code
    svn checkout http://svn.apache.org/repos/asf/lucene/solr/trunk apache-solr-1.3.dev
  6. Uncompress all of the software packages.

Create Environment Variables

  1. In Unix variants, update your .bash_login script to include the following. Of course, you'll need to set the variables correctly according to where you uncompressed the software. And don't forget to source the .bash_login file after you have changed it.
    export SUPPORT_DIR=/home/medined/support
    export LOCAL_LUCENE_HOME=$SUPPORT_DIR/locallucene-r1.5
    export LOCAL_SOLR_HOME=$SUPPORT_DIR/localsolr-r1.5
    export SOLR_HOME=$SUPPORT_DIR/apache-solr-1.3-dev
    export SOLR_CONFIG=/home/medined/.solr
    export TOMCAT_HOME=$SUPPORT_DIR/apache-tomcat-6.0.16
    
    export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=$SOLR_CONFIG"

Create Apache Solr War

  1. cd $SOLR_HOME
  2. ant dist

Configure Local Solr

  1. cp -R $SOLR_HOME/example/solr $SOLR_CONFIG
  2. cp $SOLR_HOME/libs/solr/apache-solr-1.3-dev.war $TOMCAT_HOME/webapps/solr.war mkdir $SOLR_CONFIG/lib
  3. cp $LOCAL_LUCENE_HOME/dist/locallucene.jar $SOLR_CONFIG/lib
  4. cp $LOCAL_SOLR_HOME/dist/localsolr.jar $SOLR_CONFIG/lib
  5. cp $LOCAL_LUCENE_HOME/lib/gt2-referencing-2.3.1.jar $SOLR_CONFIG/lib
  6. cp $LOCAL_LUCENE_HOME/lib/geoapi-nogenerics-2.1-M2.jar $SOLR_CONFIG/lib
  7. cp $LOCAL_LUCENE_HOME/lib/jsr108-0.01.jar $SOLR_CONFIG/lib
  8. Add the following to the end of $SOLR_CONFIG/conf/solrconfig.xml (just before the closing CONFIG tag:
      <updateRequestProcessor>
        <factory name="standard" class="solr.ChainedUpdateProcessorFactory" default="true">
          <chain class="com.pjaol.search.solr.update.LocalUpdateProcessorFactory">
            <str name="latField">lat</str>
            <str name="lngField">lng</str>
            <int name="startTier">9</int>
            <int name="endTier">17</int>
          </chain>
          <chain class="solr.LogUpdateProcessorFactory" >
           <!-- <int name="maxNumToLog">100</int> -->
          </chain>
          <chain class="solr.RunUpdateProcessorFactory" />
        </factory>
      </updateRequestProcessor>
    
     <requestHandler name="geo" class="com.pjaol.search.solr.LocalSolrRequestHandler">
        <!-- Custom latitude longitude fields, below are the defaults if not otherwise
        specified -->
         <str name="latField">lat</str>
         <str name="lngField">lng</str>
      </requestHandler>
  9. Add the following to the fields tag of $SOLR_CONFIG/conf/schema.xml
    <field name="lat" type="sdouble" indexed="true" stored="true"/>
       <field name="lng" type="sdouble" indexed="true" stored="true"/>
       <dynamicField name="_local*" type="sdouble" indexed="true" stored="true"/>

Controlling Tomcat

  • $TOMCAT_HOME/bin/startup.sh - this command starts Tomcat.
  • tail -f $TOMCAT_HOME/logs/catalina.out - this command lets you watch Tomcat's output log.
  • $TOMCAT_HOME/bin/shutdown.sh - this command stops Tomcat.

If you're lucky enough to be using Unix then combine the first two commands onto one line:

$TOMCAT_HOME/bin/startup.sh; tail -f $TOMCAT_HOME/logs/catalina.out

Importing Data Into Apache Solr

When LocalSolr is deployed into Tomcat, the default port is 8080. However, the Apache Solr import tools use a different hardcoded port. This causes me a but of angst until I realized that I could easily copy the SimplePostTool and change the port. So copy $SOLR_HOME/src/java/org/apache/solr/util/SimplePostTool.java, change the port specified on line 46 (see below) and compile it.

public static final String DEFAULT_POST_URL = "http://localhost:8080/solr/update";

I create a data file that looked like this

<add>
 <doc>
   <field name="id">01</field>
   <field name="name">HOUSE01</field>
   <field name="lat">39.36</field>
   <field name="lng">-77.4027</field>
   <field name="text">zxy</field>
 </doc>
 <doc>
   <field name="id">02</field>
   <field name="name">HOUSE02</field>
   <field name="lat">38.36</field>
   <field name="lng">-77.4027</field>
   <field name="text">zxy</field>
 </doc>
</add>

Then I simply executed the SimplePostTool program passing it the name of the data file as the program argument.

Apache Solr Admin Screen

With Tomcat running, you should be able to connect with http://localhost:8080/solr/admin/

Local Solr GIS Query

With Tomcat running, you should be able to execute a GIS-based query by connecting to http://localhost:8080/solr/select?&qt=geo&lat=38.8700&long=-77.4027&q=zxy&radius=1. The q=zxy tells Apache Solr to return all documents. The lat and long parameters indicate the center of the circle to search using decimal degrees. While the radius parameter indicates the radius, in miles, of the circle.

Good Luck!