Create Generic, Python and Docker Package Repositories with JFrog Artifactory

This JFrog tutorial will deeply educate you on how to create local, remote, and virtual repositories for Generic, Python, and Docker Package types with easy examples for your quick understanding:

In the previous tutorials of the JFrog Artifactory series, we learned about the different types of repositories, Property sets, and examples of usage of JFrog Artifactory using the Maven Package type.

In this tutorial, we will look at how to use the Generic and Docker Package type.

JFrog Artifactory supports creating a repository of Generic package type which does not have any specific type associated. You can upload any kind of artifact to the repository of generic types.

JFrog Artifactory supports creating a repository of Docker package types which helps to store built Docker images and also pull images from the external Docker Hub registry.

=> Series of Simple JFrog Tutorials

Generic, Python, and Docker Package Repositories

Generic Package Type

Create Local Repository

Local repository is a central location to upload and store your artifacts generated by builds. To create a local repository as an Admin user, select Administration TAB -> Repositories. Now click on + Add Repository in the top right and select Local Repository.

Select Generic package type.

Provide a Repository Key and click on Create Local Repository.

Once the generic package-type repository is created, we will look at how to upload and download files using the generic repository.

Click on Set Me Up and look at the CURL commands to upload and download files.

Upload files example

Download files example

The API key can be copied and saved from the user profile. See part 1 of the series for the same.

Docker Package Type

Create Local Repository

Local repository is a central location to store your artifacts generated by builds. To create a local repository as an Admin user, select Administration TAB -> Repositories. Now click on + Add Repository in the top right and select Local Repository.

Select Docker package type.

Enter the repository key and click on Create Local Repository.

Create Remote Repository

Remote repository is the cache of a repository that is managed remotely. Let’s look at how to create a Maven remote repository.

As an Admin user, select Administration TAB -> Repositories. Now click on + Add Repository -> Remote Repository in the top right.

Select Docker.

Provide the repository key and leave the remote URL as is. Also, enter Docker Hub credentials and click on Test. You will see a successful connection message at the top.

Click on Create Remote Repository. The remote repository should be now available in the list.

Create Virtual Repository

A virtual repository is a combination of both local and remote repositories having a common endpoint. To create a Virtual repository as an Admin user, select Administration TAB -> Repositories. Now click on + Add Repository ->Virtual Repository in the top right. Select Docker.

Select Docker.

Provide a repository key and scroll down to add both local and remote repositories.

Scroll down and set the Default Deployment repository as the docker local repository.

Click on Create Virtual Repository. The virtual repository should now be available.

Also Read =>> Docker Installation and Terminology

JFrog Artifactory Docker Example

In this section, we will look at how to pull docker images from the remote repository and push docker images to the local repository.

To set up, a user goes to Application TAB -> Artifactory -> Artifacts. Search for the Virtual repository and click on Set Me Up.

Let’s start with the docker login as given on the above screen. Enter your Artifactory username and password.

Let us pull a few docker images by running the following commands:

Pull Docker image format:

docker pull vniranjan25.jfrog.io/niranjan-docker-virtual/<DOCKER_IMAGE>:<DOCKER_TAG>

Example:

docker pull vniranjan25.jfrog.io/niranjan-docker-virtual/httpd
docker pull vniranjan25.jfrog.io/niranjan-docker-virtual/jenkins/jenkins
docker pull vniranjan25.jfrog.io/niranjan-docker-virtual/ubuntu

All three docker images are now downloaded using the Artifactory remote repository which is part of the Virtual repository.

Build Docker images using Dockerfile Push to Artifactory local repository:

Sample Dockerfile

FROM vniranjan25.jfrog.io/niranjan-docker-virtual/httpd
COPY index.html /var/www/html

Run the docker build and push the image:

docker build -t vniranjan25.jfrog.io/niranjan-docker-virtual/httpdexample  .

Tag the image:

docker tag vniranjan25.jfrog.io/niranjan-docker-virtual/<DOCKER_IMAGE>:<DOCKER_TAG>
docker tag e488cd6a605a vniranjan25.jfrog.io/niranjan-docker-virtual/httpdexample:v1

Push the image:

docker push vniranjan25.jfrog.io/niranjan-docker-virtual/<DOCKER_IMAGE>:<DOCKER_TAG>
docker push vniranjan25.jfrog.io/niranjan-docker-virtual/httpdexample:v1

All the above commands are run and shown below is the output:

Look at the Docker image uploaded in Artifactory.

Pypi Package Type

Suggested Reading =>> Python Tutorial Series for Beginners

Create Local Repository

JFrog Artifactory supports PyPI repositories by providing the following features:

• Ability to install PyPI packages from Artifactory
• Access to remote PyPI repositories (such as https://pypi.org/) through Remote Repositories
• Access multiple PyPI repositories through Virtual repositories which provide a single endpoint

To create a local repository as an Admin user, select Administration TAB -> Repositories. Now click on + Add Repository in the top right and select Local Repository.

Select Pypi package type.

Enter a Key and click on Create local repository.

Create Remote Repository

Remote repository is the cache of a repository that is managed remotely. Let’s look at how to create the Pypi remote repository.

As an Admin user, select Administration TAB -> Repositories. Now click on + Add Repository -> Remote Repository in the top right.

Select the Pypi package type. Provide a repository key, ensure all details are as below, and click on Create remote repository.

Create Virtual Repository

A virtual repository is a combination of both local and remote repositories having a common endpoint. To create a Virtual repository as an Admin user, select Administration TAB -> Repositories. Now click on + Add Repository -> Virtual Repository in the top right.

Select Pypi. Enter the Repository key and scroll down to add both local and remote repositories. Also, provide the local repository as a Deployment repository.

Click on Create Virtual repository.

JFrog Artifactory Python Example

Pre-requisite: Download and install the latest python3 and pip on your system.

Create Python package

Create a folder called ~/pypi_pkg/pythonex and create a new file called pymultiply.py . Add the content below.

def multiply_number(a,b):
         result=a*b;
         return result;

Add one more file called hello.py with the content below.

def Helloworld():
          print("Hello World from Artifactory")
Helloworld()

This file is needed to treat the directory containing files as packages.

from pythonex.pymultiply import multiply_number
from pythonex.hello import Helloworld

Create the setup.py file one level up in the package folder.

from setuptools import setup
setup(
    name='pythonex',
    version='0.12',
    description='Testing the installation of package'
    license='MIT',
    author='Niranjan',
    author_email='vniranjan1225@gmail.com',
    url='#',
    packages=['pythonex']
    zip_safe=False)

Deploy the Pypi package

Click on Set Me Up in the Virtual repository and setup .pypirc file. To deploy packages using setuptools we need to add the Artifactory URL to this file. It is located in the home directory.

Add the content to the .pypirc file.

Now cd to the package folder where the setup.py file is located and run the command.

python3 setup.py bdist_wheel upload -r local

Look at the local repository in Artifactory.

Resolve/Install the deployed Pypi package

The deployed Pypi package in Artifactory is installed below.

Test the python package

Create a file test.py in ~/pythontest directory and add the following contents.

from pythonex import multiply_number
x=multiply_number(10,10)
print ("Multiplication of 10 and 10: ", x)

Run the test.py file

$ python3 test.py

Working With Corporate Proxy

Normally, corporate connectivity to public sites requires setting up a proxy with or without authentication.

In the above example of Generic and Docker package type before running the CURL and Docker commands the proxy commands should be run in the command line or can be set up in the profile file.

Before the proxy can be set you should request your proxy admin to whitelist the JFrog Artifactory SAAS URL for the proxy server.

On Linux

$ export http_proxy=http://proxy-server-ip:port
$ export http_proxy=http://proxy-server.mycompany.com:8080
OR with authentication
$ export http_proxy=http://username:password@proxy-server.mycompany.com:8080

On Windows

$ set http_proxy=http://proxy-server-ip:port
$ set http_proxy=http://proxy-server.mycompany.com:8080
OR with authentication
$ set http_proxy=http://username:password@proxy-server.mycompany.com:8080

Post whitelisting and setting up the proxy, you should be able to perform CURL, pip, and Docker commands.

To set up a proxy for Maven add the proxies tag in the ~/.m2/settings.xml file.

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <proxies>
    <proxy>
      <id>mycompanyproxy</id>
      <active>true</active>
      <protocol>http</protocol>
      <username>proxyuserid</</username>
      <password>proxypassword</password>
      <host>proxy.mycompany.com</host>
      <port>80</port>
      <nonProxyHosts>localsite.com|somelocalsite.company.com</nonProxyHosts>
    </proxy>
     </proxies>
 </settings>

After adding the above steps, you should be able to run mvn build and deploy commands.

Conclusion

In this tutorial, we looked at the Generic and Docker package types. This article focused on both of the package types with an example of both upload and download.

In the next tutorial, we will look at how to use JFrog Artifactory for NPM Package type and how to use the Artifactory plugin with Jenkins and its example of maven, npm, and docker.

PREV Tutorial | NEXT Tutorial