Maven Dependency Resolution - A Repository Perspective

Dependencies and repositories are central to Maven. Most people approach Maven as a tool in a very large stack of technology and they might not have the time to dig into the details. You might be using Eclipse, Maven, Nexus, Hudson, in addition to the various libraries and servers that are involved in your development environment, and Maven's dependency management is often so effective at hiding the details of repositories and dependencies that you take some of the complexity behind the scenes for granted. This post explains how dependencies and repositories work together for people who are interested some of the details that make dependency management in Maven "just work".

First of all, we need to know the default repository layout, which affects the repository path of each artifact. The related Maven source code is easy to understand, here it is:

  private static final char PATH_SEPARATOR = '/';

  private static final char GROUP_SEPARATOR = '.';

  private static final char ARTIFACT_SEPARATOR = '-';

  public String pathOf( Artifact artifact )
  {
    ArtifactHandler artifactHandler = artifact.getArtifactHandler();

    StringBuilder path = new StringBuilder( 128 );

    path.append( formatAsDirectory( artifact.getGroupId() ) ).append( PATH_SEPARATOR );
    path.append( artifact.getArtifactId() ).append( PATH_SEPARATOR );
    path.append( artifact.getBaseVersion() ).append( PATH_SEPARATOR );
    path.append( artifact.getArtifactId() ).append( ARTIFACT_SEPARATOR ).append( artifact.getVersion() );

    if ( artifact.hasClassifier() )
    {
      path.append( ARTIFACT_SEPARATOR ).append( artifact.getClassifier() );
    }

    if ( artifactHandler.getExtension() != null && artifactHandler.getExtension().length() > 0 )
    {
      path.append( GROUP_SEPARATOR ).append( artifactHandler.getExtension() );
    }

    return path.toString();
  }

  private String formatAsDirectory( String directory )
  {
    return directory.replace( GROUP_SEPARATOR, PATH_SEPARATOR );
  }

Take TestNG as an example, assume the artifact coordinate is groupId=org.testng, artifactId=testng, version=4.8, classifier=jdk15, and packaging=jar, the above code runs like this:

1. Calcuate the repository path from the groupId, replace the groupId separator dot with path separator slash, and append another path separator. So, org.testng becomes org/testng/.
2. Append the artifactId and a path separator. Since in this example the artifactId is testng, so the path becomes org/testng/testng/.
3. Append the artifact's base version and a path separator. For release artifacts, base version is the version, for snapshots, timestamp will be converted to SNAPSHOT to get the base version. In this example, the base version is 5.8, so the path becomes org/testng/testng/5.8/.
4. Append the artifactId, a artifact separator hyphen, and the version. Now, the path becomes org/testng/testng/5.8/testng-5.8.
5. If the artifact has a classifier, append a artifact separator and the classifier. In this example the classifier is jdk15, so the path becomes org/testng/testng/5.8/testng-5.8-jdk15.
6. Finally, get the artifact extension from artifactHandler. Each packaging type has its own artifactHandler, so the extension is determined by packaging, for default packaging type jar, the extension is also jar. Append a groupId separator dot, and the extension. So the path is finally org/testng/testng/5.8/testng-5.8-jdk15.jar.

Now we know how Maven gets the repository path from artifact coordinate, but this is not the whole story. What if the artifact does not exist in local repository? what if the version of artifact is SNAPSHOT? How does Maven know where and how to get the latest SNAPSHOT?

When Maven resolves a dependency from a repository, these are the basic steps it follows:

1. If the scope of this dependency is system, simply resolve it from local file system.
2. Try to find the dependency from local repository, resolve it if it's found.
3. If the version of the dependency is explicit and non-SNAPSHOT, iterate though the remote repositories to download it, and resolve it if it's found.
4. (For version RELEASE or LATEST) When it's required, iterate all the remote repositories to get groupId/artifactId/maven-metadata.xml and merge them with the local one.
5. (For version RELEASE or LATEST) Use the merged metadata to construct an explicit non-snapshot version, resolve it if it's found locally, otherwise download it from remote repository.
6. (For SNAPSHOT version) When it's required, iterate though the remote repositories to get groupId/artifactId/version/maven-metadata.xml and merge them with the local one.
7. (For SNAPSHOT version) Use the merged metadata to construct an explicit snapshot version, resolve it if it's found locally, otherwise download it from remote repository.
8. (For SNAPSHOT version) If the resolved snapshot version is of timestamp style, like 1.4.1-20091104.121450-121, copy the file to a non-timestamp style, like 1.4.1-SNAPSHOT, and use this file.

A few notes:

In step 4 and 6: 'when it's required' means release/snapshot must be enabled for the repository, and repository updatePolicy is satisfied, or force update is enabled by -U options.

In step 5: In maven 2, LATEST will be constructed to the real latest version, no matter it's snapshots or not, but in maven 3, only the latest non-snapshot version will be used.

in step 4: a typical groupId/artifactId/maven-metadata.xml is like this:

<!--?xml version="1.0" encoding="UTF-8"?-->

  org.sonatype.nexus
  nexus

    1.4.2-SNAPSHOT
    1.4.0

      1.3.5
      1.3.6
      1.3.7-SNAPSHOT
      1.4.0-SNAPSHOT
      1.4.0
      1.4.0.1-SNAPSHOT
      1.4.1-SNAPSHOT
      1.4.1.1-SNAPSHOT
      1.4.2-SNAPSHOT

    20091214221557

All the available versions in this directory are listed in order. The latest element points to the latest version, and the release element points to the latest release version. With the help of this file, Maven knows what the latest release version is.

In step 6: a typical groupId/artifactId/version/maven-metadata.xml is like this:

<!--?xml version="1.0" encoding="UTF-8"?-->

  org.sonatype.nexus
  nexus
  1.4.2-SNAPSHOT

      20091214.221414
      13

    20091214221558

The timestamp and buildnumber of the latest snapshot is listed, so Maven can understand what the latest snapshot artifact file is. In this case, it is nexus-1.4.2-20091214.221414-13.pom.

Now that you are familiar with the process Maven uses to resolve dependencies, calculate dependency paths, and resolve conflicts you can refer to this post whenever you have an unexpected failure resolving a dependency.