apt

The apt reposiroty is the wan on Ubuntu that you add extra software features. Usually this requires the internet - which of course is an issue.

In order to get this too work in the office we need 3 things

  • internet connection
  • apt-mirror
  • Usb Drive
  • Intranet
    • Web Server
    • Modified sources.list

apt-mirror

To install and configure

$ apt-get install apt-mirror
$ nano /etc/apt/mirror.list
$ sudo apt-mirror   

The mirror.list contains the branches of the apt repos that you wish to track.

mirror.list

This is what this file looks like

# apt-mirror configuration file
##
## The default configuration options (uncomment and change to override)
#
#
set base_path /media/STORAGE/
# set mirror_path $base_path/mirror
# set skel_path $base_path/skel
# set var_path $base_path/var
#
# set defaultarch <running host architecture>
set nthreads 20

#
# 8.04 "hardy" amd64 mirror
deb-amd64 http://us.archive.ubuntu.com/ubuntu hardy main restricted universe 
multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu hardy-updates main restricted 
universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu hardy-security main restricted 
universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu hardy-backports main restricted 
universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu hardy-proposed main restricted 
universe multiverse

You will need to make sure you get the correct ubuntu version (this is for 8.02 !!)

I also have assumed that your External or portable drive is at /media/STORAGE

You now need to run this - but be warned this will take aprox 40 Gb of disk space.

running

To run this simply do

apt-mirror

You possibly want to have this as a crontab job, if you are looking to keep an up-to-date list.

USB Drive

You now need to move the USB Drive to your production server - which has

  • 50Gb disk space
  • Web Server running

Copy the data

Do something like

rsync -xavn /media/STORAGE/ /var/www/

That places the data files in your default Web browser directory.

sources.list

You now need to edit sources.list which was looking like this

deb http://ubuntu.us.com/ubuntu hardy main restricted universe multiverse
deb http://ubuntu.us.com/ubuntu hardy-updates main restricted universe multiverse
deb http://ubuntu.us.com/ubuntu hardy-security main restricted universe multiverse

To this

deb http://192.168.0.10/ubuntu hardy main restricted universe multiverse
deb http://192.168.0.10/ubuntu hardy-updates main restricted universe multiverse
deb http://192.168.0.10/ubuntu hardy-security main restricted universe multiverse

If you are

  • Lazy
  • Sneaky

You could create a fake DNS record for

 ubuntu.us.com in A 192.168.0.10

CLient Update

The client machines now need to do

sudo apt-get update 

If you get errors - it is due to either

  • web browser setup
  • sources.list

Perl

Perl is a nice development language - especially suited to text processing.

Like most modern languages it is improved by using extra libraries and modules.

But how to do this....

Download CPAN

Perls modules come from CPAN. You need to do these steps on an internet connected machine

Installing a local CPAN repository

I maintain a local mirror of CPAN. This makes installing our web application significantly faster.

$ sudo perl -MCPAN -e shell
$ cpan> install CPAN::Mini
#Create a ~/.minicpanrc file:

# to set up our local CPAN

local: /media/STORAGE/CPAN/
remote: http://mirror.rit.edu/CPAN/
exact_mirror: 1

Set up your CPAN to use the local mirror:

$ cpan
cpan> o conf urllist file:///media/STORAGE/CPAN/
cpan> o conf commit

Cron

#Set up the minicpan to run once a day:
#crontab
3 * * * minicpan

This will take some time to get the library's - and consume aprox 30Gb of data.

When it is all done - remove the Drive and move to Server.

Intranet CPAN Server

We need to move the files to the web server - and make sure that they are accessable.

rsync -xavn /media/External/CPAN/ /var/www/CPAN

Then on the machines that need CPAN

$ cpan
cpan> o conf urllist file:///var/www/CPAN/
cpan> o conf commit

Python PIP

Python like APT and Perl also has modules.

This is how I made my own local PIP repository

Internet server

Goto Pypi and scape the Python 3.3 module list.

Place all the modules in a file ... which starts like this

115wangpan  Unofficial Python API wrapper for 115.com
2factorcli  This is a simple python program to allow you to store and generate time-based one-time passwords in a GPG encrypted vault.
2gis    2gis library for Python
2lazy2rest  Effortless generation of PDF, HTML & ODT documents from RST (ReStructuredText)
3d-wallet-generator A tool to help you design and export 3D-printable bitcoin/cryptocurrency wallets
3to2    Refactors valid 3.x syntax into valid 2.x syntax, if a syntactical conversion is possible
42cc-pystyle    flake8 checks for 42 Coffee Cups style guide
51degrees-mobile-detector-v3-trie-wrapper   51Degrees Mobile Detector (C Trie Wrapper).
51degrees-mobile-detector-v3-wrapper    51Degrees Mobile Detector (C Pattern Wrapper).
ABN Validate Australian Business Numbers.

I want just the first field ....

cat t.t | cut -f 1 | sort | head

Basket

We need to install basket

venv34  (I am doing this in a venv environment)
pip install basket

Init

basket init

By default it will store in ~/.basket

Test 1 package to download

basket download nose coverage
Added coverage 4.0a6.
Added nose 1.3.7.

Check

(venv) bash-3.2$ ls ~/.basket/
coverage-4.0a6.tar.gz   nose-1.3.7.tar.gz

Massdownload

Another 30 Gb of disk space here needed - Per Python Version !!!

cat t.t | cut -f 1 | sort | parallel -j 20 basket {}

Using these files

  • Copy the ~/.basket to a central location
  • Make sure accessable via NFS say
#To install pyramid
pip install --no-index -f file:///path/to/.basket pyramid