Search

Language:  
Search for:

Available article translations:

Plesk Mirror Setup Tool v2.0

Article ID: 113337, created on Feb 20, 2012, last review on Aug 13, 2015

APPLIES TO:
  • Plesk 12.0 for Linux
  • Plesk 11.0 for Linux
  • Plesk 11.5 for Linux
  • Plesk 10.3 for Linux/Unix
  • Plesk 12.0 for Windows
  • Plesk 11.5 for Windows

Symptoms

It is required to have a local Plesk repository (mirror) for use in an internal network environment.

A mirror - is a web site available via HTTP, which can be used as a source for installing/upgrading Plesk family products using autoinstaller. A mirror can be used in either of the following ways:

  • by changing DNS resolving and setting up the mirror web host so that http://autoinstall.plesk.com (for Plesk Linux) or http://autoinstall-win.pp.parallels.com/ (for Windows) would point to the mirror;
  • by specifying the URL to the mirror in the SOURCE variable of the /root/.autoinstallerrc file on the machine where autoinstaller is launched;

    [root@container ~]# cat  /root/.autoinstallerrc
    SOURCE=http://uri.of.local.mirror/here/   
    
  • by specifying the mirror URL in the --source option when launching autoinstaller in interactive mode.

The main feature set of the Mirror Setup Tool allows providers to manage selected Plesk versions and operating systems. The tool also offers an interface for managing pre-launch testing of upcoming Plesk releases.

The tool should be launched on Linux system. The mirror (rsync) can be running on Linux or Windows systems.

Please feel free to discuss the tool in the special thread on the Plesk Forum.

IMPORTANT NOTE:

Plesk Windows and Plesk Linux mirrors CAN NOT be located in the same folders. Mirrors MUST be created in separate paths.

Prerequisites:

  1. Virtual host used for mirroring should have all script handlers (php, pl, cgi, py, fcgi etc) disabled;
  2. Virtual host or directory used for mirroring should have option +FollowSymLinks set;
  3. Unless the other sources are specified, rsync://rsync.autoinstall.plesk.com/autoinstall and rsync://autoinstall-win.pp.parallels.com/autoinstall for Plesk Windows must be accessible from the machine, where the script is being run, and from the destination machine;
  4. PHP must be compiled with support of the below extensions on the machine, where the script is being run (see http://www.php.net/manual/en/ssh2.installation.php):

    • ssh2_connect
    • simplexml
    • XML DOM

Below are sample instructions on compiling PHP with the required extensions (they can be different depending on the operating system version):

CentOS 5:

    # yum install php-cli php-devel php-pear php-xml openssl-devel gcc
    # wget http://libssh2.org/download/libssh2-1.4.0.tar.gz
    # tar -xzf libssh2-1.4.0.tar.gz
    # cd libssh2-1.4.0
    # ./configure
    # make install all
    # pecl install -f ssh2
    # echo "extension=ssh2.so" >> /etc/php.d/ssh2.ini
    # /etc/init.d/httpd restart

CentOS 6:

     #yum install gcc php-devel php-pear libssh2 libssh2-devel make
     #pecl install -f ssh2
     #echo extension=ssh2.so > /etc/php.d/ssh2.ini
     #service httpd restart
     #php -m | grep ssh2 

Ubuntu 10.04

    # apt-get install libssh2-1 libssh2-php php5-cli

Debian 5.0

    # apt-get install libssh2-1 libssh2-1 php5-cli php-pear php5-dev
    # wget http://libssh2.org/download/libssh2-1.4.0.tar.gz
    # tar -xzf libssh2-1.4.0.tar.gz
    # cd libssh2-1.4.0
    # ./configure
    # make install all
    # echo "extension=ssh2.so" >> /etc/php5/conf.d/ssh2.ini
    # /etc/init.d/apache2 restart

Resolution

Download the tool using the following link: mirrorctlv2.0.zip.

User Interface (Syntax)

Note: for Plesk for Windows mirroring specify --win option for all commands used (see below); Use separate configuration files for Plesk Windows and Plesk Linux mirrors. If the below descriptions are not clear, try reading the "Use Case Examples" section below.

Options for working with an external source repository:

show-all-releases [--win] [--src-host=<URL to source repository>] - list available product versions.
product-list - obsoleted by show-all-releases option. Not shown in help anymore.
os-list --release=<product release> - list available OSes for the selected product version.

Options for configuring local repository:

cfg --add [--win] --release=<product release> --os=<product os> [--cfg-file=<path to config file>] - add OS for selected version of product to repository.
cfg --remove [--win] --release=<product release> --os=<product os> [--cfg-file=<path to config file>] - remove OS for selected version of product from repository.
cfg --check [--win] [--cfg-file=<path to config file>] - check validity of config file, including compliance of the required product versions and operating systems with their presence in 'plesk.inf3' file.
cfg --import --path=<repository path> - get configuration from local repository.
cfg --export --path=<repository path> - save configuration to local repository

Options for synchronizing the local repository:

update [--win] --path=<repository path> - sync current state of repository according to the changes in configuration

Note: The update command works only with the default repository ( rsync://autoinstall.plesk.com ). If you need to synchronize your mirror with another mirror you need to use the push command below.

push [--win] --src-host=<URL to source reposity> --path=</repository/path> [--cfg-file=<config>] [--private-key=<key> --public-key=<key>] - merge local repository to main repository and apply changes to configuration of main repository.

Note: To avoid confusion when synchronizing with main repository, the configuration file used for the push command is recommended to be different from the one used for the update command.

Common options:

version – display the tool version
help, -h, --help – list available commands

Miscellaneous options:

--path: is used to specify path to the receiving host. May include login, password, hostname, port and path in the file system

Format:

    --path=[login[:password@][host:][port]/path/to/repository   

Note: The path must exist, and the user with specified login must have "rwx" access to it.

Examples: Using all parameters:

 --path=root:blabla@server.domain.com:344/path/to/repository

Specifying username:

--path=root@server.domain.com:/path/to/repository

Specifying port. Current username will be used along with corresponding ssh keys:

 --path=server.domain.com:4322/path/to/repository

Specifying username and password. Default port will be used:

--path=root:blabla@server.domain.com:/path/to/repository

Specifying local repository:

--path=/path/to/repository

--src-host - used to specify parameters for a source repository in the format of rsync command (see "man rsync").

Format:

Using rsync daemon as a source:

--src-host=rsync://[user@]host[:port]/path

Note: When using rsync protocol, start rsyncd daemon on the source host and configure it to be accessible from both the target server and the server where the tool is being run.

Get data through an SSH connection:

--src-host=[user@]host:/path   

Note: When using SSH protocol for synchronizing data from the specified repository you need to set up authorization via SSH keys so that the user specified in --path option could login without a password to the source host as a user specified in the --src-host option. Additionally, the user under which the tool is being run on the local machine should able to be authorized on the needed host without password using SSH keys as a user specified in --src-host option.

Get data from the local disk:

--src-host=/path

Note: Local source host for push command can be specified only in combination with local --path (i.e. when source and destination repositories are physically located on the same machine).

Workflow

  1. Download the archive with tool.

  2. Unzip it.

  3. Select required products and OSes that you would like to mirror. You may use the show-all-releases command to view available products and os-list command to view OSes supported by particular product.

  4. Create configuration of repository on local server with cfg --add and cfg --remove options.

  5. Save the current configuration of the repository using cfg --export option. It will be saved to a mirror.xml file on the specified path.

  6. Use cfg --import option to load configuraion which was saved earlier, or configuration of an existing mirror, which was created using mirrorctl utility and, then, saved to `path/mirror.xml.

  7. Create a mirror for needed product versions and OSes using update command. Note, that from this repository you will be able to install only the product versions and OSes which were specified in the mirror configuration. All other versions will not be available in this repository for autoinstaller utility.

  8. If you need to mirror a server different from the official repository, you may use the push command. Using that command you can upload product versions and OSes specified in a configuration file to an existing mirror, specified using --path. (This command may work incorrectly with a mirror, which was created _not_ using mirrorctl tool).

Use Case Examples

Common precondition:

Download the latest available version, unzip it and go to the mirrorctl directory:

# wget --output-document=/opt/mirrorctl.zip http://kb.odin.com/Attachments/kcs-20816/mirrorctlv2.0.zip
# unzip /opt/mirrorctl.zip
# cd /opt/mirrorctl

Use Case 1 - Creating a mirror for a limited list of product versions/OSes

I would like to create a mirror where only these product versions/OSes are available:

PP 10.4.4 for Deb 5.0 i386 & Ubuntu 8.04 i386,
PPP 9.5.4 for CentOS5 x86_64,
SiteBuilder 4.5 for Ubuntu 8.04 i386
PP 10.4.4 for Windows

Instructions:

1) Create mirror configuration by adding the needed OSes for the needed product versions:

$ ./mirrorctl cfg --add --release=PLESK_10_4_4 --os=deb5.0_i386
$ ./mirrorctl cfg --add --release=PLESK_10_4_4 --os=ubt8.04_i386
$ ./mirrorctl cfg --add --release=PLESK_9_5_4 --os=cos5_x86_64
$ ./mirrorctl cfg --add --release=SB_4_5_0 --os=ubt8.04_i386
$ ./mirrorctl cfg --add --win --release=PANEL_10_4_4_WIN --cfg-file=./plesk_win_mirror

2) Create a directory for the mirror:

$ mkdir /var/www/html/mirror/
$ mkdir /var/www/html/mirror_windows/

3) Synchronize the mirror with the official repository:

$ ./mirrorctl update --path=/var/www/html/mirror/
$ ./mirrorctl update --win --path=/var/www/html/mirror_windows/ --cfg-file=./plesk_win_mirror

4) Check the list of releases available in the created mirror:

$ ./mirrorctl show-all-releases --src-host=/var/www/html/mirror/
$ ./mirrorctl show-all-releases --win --src-host=/var/www/html/mirror_windows/

5) Check the list of mirrored OSes for a particular release:

$ ./mirrorctl os-list --release=PLESK_10_4_4 --src-host=/var/www/html/mirror/

Use Case 2 - Setting up periodic updates for your mirror

I would like my mirror (where only a limited list of product versions/OSes is available) to get periodically updated with the latest product releases.

Instructions:

  1. Create mirror configuration by adding the needed OSes for the needed product versions:

    $ /home/user/mirrorctl/mirrorctl cfg --add --release=PLESK_10_4_4 --os=deb5.0_i386 --cfg-file=/var/www/html/mirror/myconfig.xml
    $ /home/user/mirrorctl/mirrorctl cfg --add --release=PLESK_10_4_4 --os=ubt8.04_i386 --cfg-file=/var/www/html/mirror/myconfig.xml
    $ /home/user/mirrorctl/mirrorctl cfg --add --release=PLESK_9_5_4 --os=cos5_x86_64 --cfg-file=/var/www/html/mirror/myconfig.xml
    $ /home/user/mirrorctl/mirrorctl cfg --add --release=SB_4_5_0 --os=ubt8.04_i386 --cfg-file=/var/www/html/mirror/myconfig.xml
    $ /home/user/mirrorctl/mirrorctl cfg --add --win --release=PANEL_10_4_4_WIN --cfg-file=/var/www/html/mirror/windows_myconfig.xml
    
  2. Configure crontab to automatically update the mirror:

    $ crontab -l
    # m h dom mon dow command
    59 23 * * * cd /home/user/mirrorctl/ && ./mirrorctl update --path=/var/www/html/mirror/ --cfg-file=/var/www/html/mirror/myconfig.xml > ./mirror-cron.log 2>&1
    59 22 * * * cd /home/user/mirrorctl/ && ./mirrorctl update --win --path=/var/www/html/mirror_windows/ --cfg-file=/var/www/html/mirror/windows_myconfig.xml > ./mirror-cron-win.log 2>&1
    

Use Case 3 - Adding OSes

I want to add a particular OS for a particular product version to my mirror.

Instructions:

  1. Import the current configuration of the repository which you earlier created using mirrorctl utility:

    $ ./mirrorctl cfg --import --path=/var/www/html/mirror/ --cfg-file=myconfig
    
  2. Add the needed OS for the needed product version:

    $ ./mirrorctl cfg --add --release=PLESK_9_5_4 --os=cos5_i386 --cfg-file=myconfig
    
  3. Synchronize the mirror:

    $ ./mirrorctl update --path=/var/www/html/mirror/ --cfg-file=myconfig
    

Use Case 4 - Downloading a particular OS/version

I want to download a particular OS for a particular product version to my mirror from the official repository without downloading other mirrored versions/OSes.

Instructions:

  1. Generate a configuration file containing only the OSes/product versions that you wish to download:

    $ ./mirrorctl cfg --add --release=PLESK_10_4_4 --os=cos5_i386 --cfg-file=./temp-config
    $ ./mirrorctl cfg --add --release=PLESK_10_4_4 --os=cos5_x86_64 --cfg-file=./temp-config
    
  2. Download the specified OSes to an existing repository using push command:

    $ ./mirrorctl push --path=/var/www/html/mirror/ --cfg-file==./temp-config
    

Use Case 5 - Creating a mirror for testing new product versions

I want to create a test mirror for a new product version, test that version and upload only that tested version (without the latest Parallels updates which I have not yet tested) to the main repository.

Instructions:

  1. Create a test mirror for all OSes of the new product version:

    $ ./mirrorctl cfg --add --release=PLESK_10_4_4 --os=all --cfg-file=./test-config
    $ ./mirrorctl update --path=/var/www/html/test-mirror/ --cfg-file=./test-config
    
  2. Test the new version;

  3. Upload the whole product version to the main repository.

    $ ./mirrorctl push --src-host=/var/www/html/test-mirror/ --path=/var/www/html/mirror/ --cfg-file=./test-config
    

Warning: If --src-host option is not used, then the new product version will be uploaded from the official repository and may be different from the one you have tested.

Use Case 6 - Creating a mirror for testing product releases for new OSes

I want to create a test mirror with a particular list of OSes for already mirrored product versions with the latest updates, test those OSes and then upload some of the tested OSes to the main repository.

Instructions:

  1. Create a test mirror where several OSes will be available for PLESK_10_4_4:

    $ ./mirrorctl cfg --add --release=PLESK_10_4_4 --os=cos5_i386 --cfg-file=./test-config
    $ ./mirrorctl cfg --add --release=PLESK_10_4_4 --os=cos5_x86_64--cfg-file=./test-config
    $ ./mirrorctl update --path=/var/www/html/test-mirror/ --cfg-file=./test-config
    
  2. Get the list of available product releases and OSes from the test mirror created above:

    $ ./mirrorctl show-all-releases --src-host=/var/www/html/test-mirror/
    $ ./mirrorctl os-list --release=PLESK_10_4_4 --src-host=/var/www/html/test-mirror/
    
  3. Generate a new configuration file containing only those releases and OSes from the test mirror which you would like to add to the main mirror:

    $ ./mirrorctl cfg --add --release=PLESK_10_4_4 --os=cos5_i386 --src-host=/var/www/html/test-mirror/ --cfg-file=push.xml
    
  4. Check the created configuration file on the availability of the releases/OSes in the test repository:

    $ ./mirrorctl cfg --check --src-host=/var/www/html/test-mirror/ --cfg-file=push.xml
    
  5. Upload the selected releases and OSes from the test mirror to the main repository:

    $ ./mirrorctl push --src-host=/var/www/html/test-mirror/ --path=/var/www/html/mirror/ --cfg-file=push.xml
    

Warning: If --src-host option is not used, then the new product version will be uploaded from the official repository and may be different from the one you have tested.

Warning: If the --cfg-file option is not specified, the default configuration file BUILD/config.xml (relatively to the directory where the tool is located) will be used. I.e. only the versions/OSes from that configuration file will be uploaded to the main repository.

Warning: If the --cfg-file option is specified incorrectly, the configuration file specified for the push command will be used.

Managing Remote Mirrors
The tool can work with remote repositories, but with certain limitations. Below you may find examples of using --path and --src-host options in cases when remote destination mirror or remote destination and source mirrors are used.

Examples of using --path option for a remote destination mirror:

  1. Using a particular user, password and host:

    --path=user:UserSecurePassword@mymirror.tld:344/var/www/html/
    --path=user:UserSecurePassword@192.168.100.100:22/var/www/html/
    --path=root:RootSecurePassword@localhost:/var/www/html/
    
  2. Using SSH keys authorization (both private and public keys are required):

    --path=192.168.100.100:/var/www/html/
    

    If default RSA pair (id_rsa & id_rsa_pub) for current user does not exist, then default DSA pair (id_dsa & id_dsa.pub) from $HOME/.ssh/ folder will be used.

    --path=user@mymirror.tld:344/var/www/html/
    
  3. Using particular SSH keys as additional options for mirrorctl utility:

    --path=root@mymirror.tld:344/var/www/html/ --private-key=~/.ssh/id_mirror --public-key=~/.ssh/id_mirror.pub
    

Examples of using --src-host option for a remote source mirror:

  1. Using rsync via SSH protocol:

    --src-host=test@192.168.100.100:/var/www/html/test-mirror/
    --src-host=mytestmirror.tld:/var/www/html/test-mirror/
    
    • In the above case it is forbidden to specify user password or SSH port (22).
    • In case --src-host option is used, the user who launches the tool must have SSH keys authorization set up for him to be able to connect to the host specified under --src-host option and under the user specified via that option.
    • If the --src-host option is used along with the --path option, then, again, the user specified in the --path option must have SSH keys authorization set up for him to be able to connect from the host specified in the --path option to the host specified under --src-host option and under the user specified via --src-host option.
  2. Using rsync daemon on the source host:

    --src-host=rsync:// mytestmirror.tld:873/testmirror
    

    Example of rsync configuration:

    # head /etc/rsyncd.conf
    [testmirror]
    path = /var/www/html/test-mirror/
    comment = Test repository
    

    .

    # cat /etc/xinetd.d/rsync
    service rsync
    {
    # disable = yes
    socket_type = stream
    wait = no
    user = root
    server = /usr/bin/rsync
    server_args = --daemon
    log_on_failure += USERID
    }
    

Configure mirror as a source for EZ templates.

The mirror created with Mirror Setup Tool can be easily configured as a source for EZ templates.

Step 1. Configure your virtual host with the mirror to be an alias for http://autoinstall.plesk.com:
For Apache just add:

ServerAlias  "autoinstall.plesk.com"

to VirtualHost section.
If domain used for mirror is controlled by Parallels Plesk Panel, just add domain alias to the domain with Web service enabled.

Step 2 Make host autoinstall.plesk.com resolving to your mirror IP.
Configure DNS severs used for resolving on your VZ nodes to resolve host autoinstall.plesk.com to your mirror IP.
Or add "<mirror IP> autoinstall.plesk.com" to the end of /etc/hosts file on your VZ nodes:

# echo  "192.168.10.10 autoinstall.plesk.com" >> /etc/hosts

Step 3. Make symlinks used by EZ templates point to the mirrored releases.
In console: change directory to the mirror root and create symlinks pointed to the mirrored releases (for example PP 10.4.4):

    # cd /var/www/html/mirror/
    # ln -s PSA_10.4.4 PSA10
    # ln -s BILLING_10.4.4 BILLING10
    # ln -s BILLING_10.4.4 PANEL_BILLING
    # ln -s SITEBUILDER_10.4.4 SITEBUILDER5
    # ln -s SITEBUILDER_10.4.4 PANEL_SITEBUILDER
    # cd ubuntu
    # ln -s PSA_10.4.4 PANEL
    # ln -s PSA_10.4.4 PSA10
    # cd ../debian/
    # ln -s PSA_10.4.4 PANEL
    # ln -s PSA_10.4.4 PSA10
    # ln -s SITEBUILDER_10.4.4 PANEL_SITEBUILDER
    # ln -s SITEBUILDER_10.4.4 SITEBUILDER5
    # ln -s BILLING_10.4.4 PANEL_BILLING
    # ln -s BILLING_10.4.4 BILLING10

Note: In some cases it is needed to add read/execute permissions to all for the mirror recursively:

# chmod -R a+rx var/www/html/mirror/

Search words:

upgrade

plesk upgrade

plesk mirror cli tool

mirrorctl can't rsync plesk

Reduce installation time

unable to upgrade



Attachments:


56797cefb1efc9130f7c48a7d1db0f0c a914db3fdc7a53ddcfd1b2db8f5a1b9c 85a92ca67f2200d36506862eaa6ed6b8 dd0611b6086474193d9bf78e2b293040 29d1e90fd304f01e6420fbe60f66f838 bd7fc88cf1b01f097749ae6f87272128 0a53c5a9ca65a74d37ef5c5eaeb55d7f ed7be2b984f9c27de1d2dc349dc19c6d a766cea0c28e23e978fa78ef81918ab8 46a8e394d6fa13134808921036a34da8 01bc4c8cf5b7f01f815a7ada004154a2 aea4cd7bfd353ad7a1341a257ad4724a f4ce8084cb23619ef5fe9428ecc388ea 2a5151f57629129e26ff206d171fbb5f e335d9adf7edffca6a8af8039031a4c7

FEEDBACK
Was this article helpful?
Tell us how we may improve it.
Yes No
 
 
 
 
 
 
Server Virtualization
- Odin Cloud Server
- Odin Containers for Windows 6.0
- Odin Virtuozzo Containers
Automation
- Odin Automation
- Odin Automation for Cloud Infrastructure
- Odin Business Automation Standard
- Odin Virtual Automation
- Odin Plesk Panel Suite
- Web Presence Builder
- Odin Plesk Automation
- Odin Small Business Panel
- Value-added Services for Hosters
- Odin Partner Storefront
Services & Resources
- Cloud Acceleration Services
- Professional Services
- Support Services
- Training & Certification