Creating a CentOS mirror site - Revision history

Ppapa at 04:00, 31 October 2011

2011-10-31T04:00:37Z

2011-10-31T03:58:53Z

2011-10-31T03:55:25Z

2011-10-31T03:03:56Z

2011-07-13T05:58:18Z

2011-07-13T05:54:02Z

2011-07-13T05:53:38Z

2011-07-13T05:47:08Z

2011-07-13T05:46:26Z

2011-07-13T05:45:14Z

@@ Line 35: / Line 35: @@
 You should now have a directory with GPG keys for all the versions and few other directories.
-.  Use wget to download the version of CentOS that you need, in my case 5.6, while excluding the ISO installer images.
+.  Use wget to download the version of CentOS that you need, in my case 6.0, while excluding the ISO installer images.
    wget -r -np -nH --cut-dirs=2\
-     --include-directories=pub/centos/5.6 \
+     --include-directories=pub/centos/6.0 \
-     --exclude-directories=pub/centos/5.6/isos http://myfavouritemirror.com/pub/centos/
+     --exclude-directories=pub/centos/3\*,pub/centos/4\*,pub/centos/5\*,pub/centos/6,pub/centos/6.0/isos \
 The new wget arguments do the following:
@@ Line 48: / Line 49: @@
 |}
-This will obviously download just the 5.6 directory and its contents, while excluding the the isos directory for this version.
+This will obviously download just the 6.0 directory and its contents, while excluding the the isos directory for this version.
 You've now created your mirror site!
@@ Line 55: / Line 56: @@
    wget -r -N -np -nH --cut-dirs=2\
-     --include-directories=pub/centos/5.6 \
+     --include-directories=pub/centos/6.0 \
-     --exclude-directories=pub/centos/5.6/isos http://myfavouritemirror.com/pub/centos/
+     --exclude-directories=pub/centos/3\*,pub/centos/4\*,pub/centos/5\*,pub/centos/6,pub/centos/6.0/isos \
 where the extra arguement is:

@@ Line 1: / Line 1: @@
-The best way to create a CentOS mirror is to use '''rsync'''.  This has been documented [http://wiki.centos.org/HowTos/CreateLocalMirror elsewhere].  However, if you are like me and you're stuck behind a corporate firewall and no one is willing to open port 873 for you, then you can do it all with '''wget''' over port 80.  Each mirror site, usually has repositories for multiple versions of CentOS.  I only wanted to mirror the repository for CentOS 5.6.  Here's how I did it.
+The best way to create a CentOS mirror is to use '''rsync'''.  This has been documented [http://wiki.centos.org/HowTos/CreateLocalMirror elsewhere].  However, if you are like me and you're stuck behind a corporate firewall and no one is willing to open port 873 for you, then you can do it all with '''wget''' over port 80.  Each mirror site, usually has repositories for multiple versions of CentOS.  I only wanted to mirror the repository for CentOS 6.0.  Here's how I did it.
 . Set up a server running apache.  I'm sure you'll figure it out somewhere else!
@@ Line 15: / Line 15: @@
    cd /export/htdocs/pub/centos
    wget -r -np -nH --cut-dirs=2 \
-   --exclude-directories=pub/centos/3\*,pub/centos/4\*,pub/centos/5\* \
+   --exclude-directories=pub/centos/3\*,pub/centos/4\*,pub/centos/5\*,pub/centos/6\* \
    http://myfavouritemirror.com/pub/centos/

@@ Line 15: / Line 15: @@
    cd /export/htdocs/pub/centos
    wget -r -np -nH --cut-dirs=2 \
-   --exclude-directories=pub/centos/3.9,pub/centos/4,pub/centos/4.3,pub/centos/4.4,pub/centos/4.5,pub/centos/4.6,pub/centos/4.7,pub/centos/4.8,pub/centos/4.9,pub/centos/5,pub/centos/5.0,pub/centos/5.1,pub/centos/5.2,pub/centos/5.3,pub/centos/5.4,pub/centos/5.5,pub/centos/5.6 \
+   --exclude-directories=pub/centos/3\*,pub/centos/4\*,pub/centos/5\* \
    http://myfavouritemirror.com/pub/centos/

@@ Line 14: / Line 14: @@
    cd /export/htdocs/pub/centos
-   wget -r -np -nH --cut-dirs=2 --exclude-directories=pub/centos/3.9,pub/centos/4,pub/centos/4.3,pub/centos/4.4,pub/centos/4.5,pub/centos/4.6,pub/centos/4.7,pub/centos/4.8,pub/centos/4.9,pub/centos/5,pub/centos/5.0,pub/centos/5.1,pub/centos/5.2,pub/centos/5.3,pub/centos/5.4,pub/centos/5.5,pub/centos/5.6 http://myfavouritemirror.com/pub/centos/
+   wget -r -np -nH --cut-dirs=2 \
 The wget arguments do the following:

@@ Line 43: / Line 43: @@
 {|
 | '''--include-directories''' || I think this is pretty obvious!
 |-
 |}

@@ Line 28: / Line 28: @@
 |-
 | '''--exclude-directories=.....''' || Comma separated list of directories you don't want to download.
 |-
 |}

@@ Line 25: / Line 25: @@
 | '''-nH''' || Disable generation of host-prefixed directories. By default, invoking Wget with ‘-r http://myfavouritemirror.com’ will create a structure of directories beginning with myfavouritemirror.com/. This option disables such behavior.
 |-
-| '''--cut-dirs=2''' || Along with the -nH argument, wget would create the directories /pub/centos/.  This gets rid of them so the data is downloaded directly into the directory you're in.
+| '''--cut-dirs=2''' || Along with the -nH argument, wget would create the directories pub/centos/.  This gets rid of them so the data is downloaded directly into the directory you're in.
 |-
 | '''--exclude-directories=.....''' || Comma separated list of directories you don't want to download.
 |-
 |}

@@ Line 23: / Line 23: @@
 | '''-np''' || Do not ascend to the parent directory when retrieving recursively
 |-
-  -nH Disable generation of host-prefixed directories. By default, invoking Wget with ‘-r http://myfavouritemirror.com’ will create a structure of directories beginning with myfavouritemirror.com/. This option disables such behavior.
+| '''-nH''' || Disable generation of host-prefixed directories. By default, invoking Wget with ‘-r http://myfavouritemirror.com’ will create a structure of directories beginning with myfavouritemirror.com/. This option disables such behavior.
-  --cut-dirs=2  Along with the -nH argument, wget would create the directories /pub/centos/.  This gets rid of them so the data is downloaded directly into the directory you're in.
+|-
-  --exclude-directories=.....  Comma separated list of directories you don't want to download.
+| '''--cut-dirs=2''' || Along with the -nH argument, wget would create the directories /pub/centos/.  This gets rid of them so the data is downloaded directly into the directory you're in.
 |}