This series of Nexus Indexer posts focuses on integrating the Nexus Indexer into your own application. If you have an application that needs to search for an artifact by GAV coordinates, or by class name, you can use the Nexus Index format and the Nexus Indexer API to very easily search and locate artifacts in any repository that creates a Nexus Index. There are four main functions that are exposed in the Nexus Indexer API.
This post will focus on Packing indexes for supplying to consumers, and updating from other index providers
In earlier posts we have shown how to index repositories and how to search these indexes. Now we want to take these indexes and deliver them somewhere else. The Nexus Indexer will take the Lucene indexes, and package up in 2 formats (legacy and current) for consumption by others. The legacy packed index (.zip file) is simply a zip of the Lucene (v2.3) index, very straightforward. The current packed index (.gz file) consists of the contents of the Lucene index, stored in our own proprietary file format that is Lucene-independent, allowing the consumers of this index to import it into any other format of their choosing (or other versions of Lucene itself).
Below I have extended my Plexus component from the previous two posts to have a new publish method:
package org.damian;
...
/**
* Sample app to show how to integrate with the nexus indexer. Note that this is a simple plexus
* component extending the SampleApp interface
*
* public interface SampleApp
* {
* void index()
* throws IOException;
*
* Set searchIndexFlat( String field, String value )
* throws IOException;
*
* Set searchIndexFlat( Query query )
* throws IOException;
*
* Map searchIndexGrouped( String field, String value )
* throws IOException;
*
* Map searchIndexGrouped( String field, String value, Grouping grouping )
* throws IOException;
*
* Map searchIndexGrouped( Query q, Grouping grouping )
* throws IOException;
*
* void publishIndex( File targetDirectory )
* throws IOException;
* }
*
* @author Damian
*
*/
@Component( role = SampleApp.class )
public class DefaultSampleApp
implements SampleApp,
Initializable,
Disposable
{
// The nexus indexer
@Requirement
private NexusIndexer indexer;
// The nexus index packer
@Requirement
private IndexPacker indexPacker;
...
public void publishIndex( File targetDirectory )
throws IOException
{
IndexPackingRequest packReq = new IndexPackingRequest( context, targetDirectory );
packReq.setCreateChecksumFiles( true );
packReq.setCreateIncrementalChunks( true );
//NOTE: There are numerous other options you can set in the index pack request
indexPacker.packIndex( packReq );
}
}
As you can see above, there is minimal code required to pack up an index (as it ultimately isn't a very complex process). After this, you will be left with numerous files in your target directory. Here is some sample code that shows what you will have after packing up the index
public void testIndexPacking()
throws Exception
{
app.index();
File publishDir = new File( getBasedir(), "target/publish/");
app.publishIndex( publishDir );
assertTrue( publishDir.exists() );
// Legacy index format
assertTrue( new File( publishDir, "nexus-maven-repository-index.zip" ).exists() );
assertTrue( new File( publishDir, "nexus-maven-repository-index.zip.sha1" ).exists() );
assertTrue( new File( publishDir, "nexus-maven-repository-index.zip.md5" ).exists() );
// Current index format
assertTrue( new File( publishDir, "nexus-maven-repository-index.gz" ).exists() );
assertTrue( new File( publishDir, "nexus-maven-repository-index.gz.sha1" ).exists() );
assertTrue( new File( publishDir, "nexus-maven-repository-index.gz.md5" ).exists() );
// properties file
assertTrue( new File( publishDir, "nexus-maven-repository-index.properties" ).exists() );
assertTrue( new File( publishDir, "nexus-maven-repository-index.properties.sha1" ).exists() );
assertTrue( new File( publishDir, "nexus-maven-repository-index.properties.md5" ).exists() );
}
Now that we have indexing/searching/packing in the bag, let's have a look at how to have your application retrieve updates to indexes supplied from remote sources.
The update process is pretty easy to use. You simply need to supply a remote URL to your IndexingContext, and that URL will be reused when retrieving updates.
Below I have extended my Plexus component again with the new update method
package org.damian;
...
/**
* Sample app to show how to integrate with the nexus indexer. Note that this is a simple plexus
* component extending the SampleApp interface
*
* public interface SampleApp
* {
* void index()
* throws IOException;
*
* Set searchIndexFlat( String field, String value )
* throws IOException;
*
* Set searchIndexFlat( Query query )
* throws IOException;
*
* Map searchIndexGrouped( String field, String value )
* throws IOException;
*
* Map searchIndexGrouped( String field, String value, Grouping grouping )
* throws IOException;
*
* Map searchIndexGrouped( Query q, Grouping grouping )
* throws IOException;
*
* void publishIndex( File targetDirectory )
* throws IOException;
*
* void updateRemoteIndex()
* throws IOException;
* }
*
* @author Damian
*
*/
@Component( role = SampleApp.class )
public class DefaultSampleApp
implements SampleApp,
Initializable,
Disposable
{
// The nexus indexer
@Requirement
private NexusIndexer indexer;
// The nexus index updater
@Requirement
private IndexUpdater indexUpdater;
...
public void updateRemoteIndex()
throws IOException
{
IndexUpdateRequest updRequest = new IndexUpdateRequest( context );
//not too much to configure with the IndexUpdateRequest, but you can
//supply your own ResourceFetcher if you would like to use some other
//means to retrieve the index file than the default. You can also
//add auth and proxy info to default ResourceFetcher, and you can set
//a transfer listener to be notified of transfer events.
//But by default, will simply use the remote index url assigned to the index
//context to retrieve new index file, and merge locally
//also you can force full update, which will wipe out what is local, and replace
//with the latest remote content.
indexUpdater.fetchAndUpdateIndex( updRequest );
}
}
As you can see above, there is minimal code required to update a remote index. I have not included test code here that will go and retrieve a remote index, but there are examples in the nexus source code where we do this ourselves. You can look at the DefaultIndexerManager in the nexus-app module for an example.
This covers the Nexus Indexer API. There are some big changes coming down the pipeline for the Nexus Indexer, so stay tuned for details!!!
A Maven project sample can be found here http://svn.sonatype.org/nexus/trunk/sandbox/nexus-indexer-sample and will be updated periodically as I put together more details for the blog posts.