Pluggable Storage

This site is the new docs site currently being tested. For the actual docs in use please go to https://www.jenkins.io/doc.

The current default approach of storing everything into the filesystem is the main reason why Jenkins does not fit the "Cloud Native" model, with features like high availability, zero downtime, or pay per use. While there are plenty of plugins that implement parts of this vision, this becomes cumbersome to configure and a usability nightmare for users, as jep:300[] has pointed out. Going towards a model where cloud services are consumed where it makes sense, the overall complexity on operating Jenkins in a cloud or containerized environment is greatly reduced. Other related projects would greatly benefit from cloud native storage for Jenkins.

There are several clear areas open for improvement. A major pain point is the usage of local disk as all-purpose storage, which causes issues running on containerized or distributed environments, requiring highly performant filesystems, and all the configuration pain like initial sizing and resizing with downtime. We believe that by using cloud provided services the overall usability, performance and scalability can be improved while enabling new demanded functionality.

You can find more information about Pluggable Storage and priorities in this blogpost.

Status summary

Below you can find a summary of ongoing activities and their current status:

Type / Status Comment Implementation(s)

Artifacts
(Available)

Fully delivered, with support for uploading artifacts directly from agents. Related JEPs: JEP-202.

Extension point

Artifact Manager on S3, Azure Artifact Manager, <More implementations>

Credentials
(Available)

Completed before the JEP process was introduced.

Extension point

Kubernetes Credentials Provider, AWS Secrets Manager Credentials Provider, <More implementations>

Build logs
(Preview / Paused)

Pipeline Log Storage API and reference implementations are available for preview, only Jenkins Pipeline job types are supported. Related JEP: JEP-210.

Jenkins core APIs and reference implementations have not been merged/released yet, but prototypes are available for evaluation. Related JEPs: JEP-207, JEP-212

Pipeline logging:

Jenkins core:

System logs
(Available)

Jenkins supports custom log appenders using standard java.util.logging configuration options.

Syslog logger, non-Jenkins implementation

Task logs
(Not started)

Storage of system logs and various tasks (e.g. agent connection or SCM polling)

N/A

Configurations
(Paused)

Largely replaced by the Configuration as Code plugin which allows storing Jenkins configurations in SCM or other locations without a database.

Related JEPs: JEP-213

N/A

Test Results
(Available)

API is available in JUnit Plugin.

Junit SQL Storage

Code Coverage results
(Not started)

Planned only for plugins based on Code Coverage API which unifies the storage implementation. See Runs for other coverage report types.

N/A

Builds/Runs
(Not started)

Storage of full build records in an external database. Includes storing build data which is not otherwise listed (such as logs or test results).

N/A

Jobs
(Not started)

Storage of Job configurations and job-specific metadata in an external database. Existing plugins like Jenkins Pipeline and JobDSL partially address this case by keeping configurations in SCM.

N/A

Fingerprints
(Preview)

Jenkins 2.252+ include the external fingerprint storage API which can be consumed by plugins. More info: GSoC Project Page

Related JEPs: JEP-226

Redis, PostgreSQL (unreleased)

Workspaces
(Not started)

Proposed as a GSoC 2019 project: Cloud Features for External Workspace Manager Plugin

N/A

The list above is not complete. Other storage types may be considered according to the feedback. You can find more information about Pluggable Storage and priorities in this blogpost.

Artifact Storage

There are many existing plugins allowing to upload and download artifacts from external storage (e.g. S3, Artifactory, Publish over SFTP, etc., etc.), but there are no plugins which can do it transparently without using new steps. In many cases the artifacts also get uploaded through the Jenkins controller, and it increases load on the system. It would be great if there was a layer which would allow storing artifacts externally when using common steps like Archive Artifacts.

Jenkins 2.118+ offers an extended jenkinsdoc:jenkins.util.VirtualFile[] API which allows implementing external artifact managers using the ArtifactManagerFactory extension point.

Implementation example(s):

Related JEPs:

Build Log Storage

Logs disk usage causes the same issues previously mentioned for artifacts. Plus an external focused log storage such as AWS CloudWatch Logs allows demanded features as centralized log management, log retention policies, advanced querying, etc. There are already options to externally ship the logs to a backend, or plugins that would do that like the aws-cloudwatch-logs-publisher-plugin, but there is no integrated way to both send and display logs from external log storage. The External log storage work is tracked as https://issues.jenkins.io/browse/JENKINS-38313.

Reference implementation(s):

Related JEPs:

Configuration Storage

Although configurations are not big, externalizing them is a critical task for getting highly-available or disposable Jenkins controllers. There are many ways to store configurations in Jenkins, but 95% of cases are covered by the XmlFile layer which serializes objects to disk and reads them using the XStream library. Externalizing these XmlFiles would be a great step forward.

There are several prototypes for externalizing configurations, e.g. in DotCI. There are also other implementations which could be upstreamed to the Jenkins core.

Related JEPs:

Credentials

In Credentials Plugin 1.15+ there is a CredentialsProvider extension point which allows referencing and resolving external credentials. This engine allows implementing external credentials for plugins implementing Credentials API..

Implementation example(s):

Other credentials API in Jenkins (like jenkinsdoc:hudson.util.Secret) are not supported.

Test results

In common CI/CD use-cases a lot of the space is being consumed by test reports. This data is stored within JENKINS_HOME, and the current storage format requires huge overheads when retrieving statistics and, especially, trends. In order to display trends, each report has to be loaded and then processed in-memory.

The main purpose of externalising Test Results is to optimize Jenkins logic by querying the desired data from specialized external storages, e.g. from Document-based databases like Elasticsearch. According to the current plan, JUnit Plugin will be extended in order to support such external storage in its APIs being widely used by test reporting plugins.

Status:

Please try it out, report issues to GitHub and general feedback to GitHub#142.

Fingerprints

The fingerprints are stored within JENKINS_HOME inside a local XML-based database. Externalizing fingerprints decreases the dependence of Jenkins on the physical disk storage of the controller, and allows configuring of cloud storages which can be cheaper, and more reliable. Another advantage is that it would allow tracing fingerprints across Jenkins instances and the entire CI/CD flow.

Status: