Binary repository manager: Difference between revisions

Revision as of 20:36, 9 February 2016

Revision as of 21:25, 18 February 2016

Line 1:

{{refimprove|date=May 2015}}

A '''binary repository manager''' is a software tool designed to optimize the download and storage of binary files used and produced in software development. It centralizes the management of all the binary artifacts generated and used by the organization to overcome the complexity arising from the diversity of binary artifact types, their position in the overall workflow and the dependencies between them.

Line 55:

* Environment variables

* Packages installed

== What is a binary repository manager?<ref>{{cite web|title=What is a Binary Repository Manager|url=http://www.jfrog.com/binary-repository/|website=www.jfrog.com|publisher=JFrog Ltd.|accessdate=15 January 2015}}</ref> ==

In common day-to-day usage, the term “binary repository” is frequently used to refer to a “binary repository manager”, however, as the term suggests, one manages and the other is managed. A binary repository manager fulfills several functions for each step in the software development lifecycle.

=== Managing multiple repositories ===

Organizations usually need to use several binary repositories in the development process. Repositories are structured according to several considerations such as organizational (project, department and access privileges), target environment (staging or production), artifact type (jar, rpm, gem etc.), artifact state (integration, release etc.) and more. A binary repository manager assigns and manages the permissions that determine which entities in the organization can access each binary repository.

=== Storing local binaries ===

Local repositories are physical, locally managed repositories into which an organization can deploy binaries. Typically, these are used to deploy internal and external releases as well as development builds, but they can also be used to store binaries that are not widely available on public repositories such as 3rd party commercial components that the organization has purchased. Using local repositories, all internal resources can be made available from a single access point across the organization from one common URL

=== Proxying and caching external binaries ===

[[File:BinaryRepositoryManagerOnDemandProxy.png|thumb|right|alt=On-demand proxy|On-demand proxy]]

[[File:BinaryRepositoryManagerMirrorProxy.png|thumb|right|alt=Mirror proxy|Mirror proxy]]

It is common for organizations to need third party artifacts hosted in an external repository. This can introduce an element of risk since an organization cannot control access to an external resource. Moreover, network latency and bandwidth can directly affect development speed, especially if the binaries used are very large. A team of developers can have progress severely hampered if its members need to download the latest build of several dependencies several times a day, where each download may take several minutes.

To overcome the risks of using an external repository, a binary repository manager maintains a caching proxy to the external resource, known as a remote repository. By using remote repositories, the binary repository manager removes the organization’s dependence on the external repository. Since external binaries are stored in a local cache, it means that they can be served rapidly to other machines on the same network after the initial request - either to developers or directly to continuous integration servers themselves.

There are two ways to proxy external repositories: on-demand, or mirrored:

*In an on-demand proxy, requests to the remote repositories happen only the first time a developer requests an artifact that is not yet cached in the proxy. Any further requests from the other developers will use the copy in the proxy repository. Therefore, the disk space and bandwidth requirements are usually low, since only artifacts that are actually used are cached.

*In a mirrored repository, all changes in the original repository are automatically synched to the mirror, so even the first request for an artifact is always resolved from the closer mirror. This is also known as repository replication. Here, however, disk space and bandwidth requirements are therefore usually higher.

=== Grouping repositories ===

When multiple teams and build tools in an organization use the same binary repository manager, the number of repositories being used can multiply very quickly since each team may have a different set of permissions or usage patterns, and may use different 3rd party resources. This can make repository setup very complex with a potentially long list of repositories to configure. To shorten the list of repositories that developers or build tools need to be familiar with, the administrator may define Virtual Repositories. A virtual repository simplifies development by encapsulating any number of local and remote repositories, and representing them as a unified repository accessed from a single URL. The binary repository manager internally optimizes how binaries are uploaded to, or downloaded from the local and remote repositories underlying the virtual repository that the developer is exposed to.

=== Supporting distributed development teams ===

[[File:Binary Repository Manager Hierarchy.png|thumb|right|alt=Hierarchy of binary repository managers|Hierarchy of binary repository managers used to replicate repositories for geographically distributed development teams]]

When teams that access repositories are in different geographical locations, a set of binary repository managers can be set up hierarchically to manage the repositories of all the distributed teams. A local repository manager is set up in each ___location to serve as a mirror of the remote server and synchronize the different repositories’ contents, and then a “Master Repository Manager” is proxied by the distributed local managers for local caching.

=== Artifact promotion ===

A binary repository manager can enforce an organization’s development workflow by setting different permissions for each repository to only allow authorized users to promote artifacts from one repository to the next one in the workflow. For example, a release candidate must go through integration testing and QA before being made available to other teams. Using the binary repository manager, only authorized members of the QA team can promote the release candidate to the releases repository once it has passed the QA process. Then, the production systems can be configured to pull artifacts only from the releases repository.

==Universal Package Manager==