HP3000-L Archives

December 2006, Week 5

HP3000-L@RAVEN.UTC.EDU

Options: Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
"John K." <[log in to unmask]>
Reply To:
John K.
Date:
Sat, 30 Dec 2006 01:50:27 -0500
Content-Type:
text/plain
Parts/Attachments:
text/plain (62 lines)
Does anyone know what the maximum number of files and/or directories you 
can put on a Data DVD (single-layer) is?  I think I may be running into a 
DVD directory limitation.

My search hasn't turned up anything, and my backup process (Roxio) appears 
to be choking on the volume of files and directories rather than the data 
volume.

Below is a description of what is being "archived."

The data consists of snapshots of web pages (made with wget).  The 
snapshots are captured every 15 minutes by a PHP script running as a "run 
at boot" task on Windows XP Pro. Most of the files are small (0k to 10k 
bytes) but there are millions of them in the course of a month.

The directory structure is:

Page_name\yyyy\mm\dd\Page_name_yyyy-mm-dd_HH-MM\files

Below is a sample listing of several sequential directory names:

page_021\2006\12\28\page_021_2006-12-28_00-00
page_021\2006\12\28\page_021_2006-12-28_00-15
page_021\2006\12\28\page_021_2006-12-28_00-30
page_021\2006\12\28\page_021_2006-12-28_00-45
page_021\2006\12\28\page_021_2006-12-28_01-00
page_021\2006\12\28\page_021_2006-12-28_01-15
page_021\2006\12\28\page_021_2006-12-28_01-30
page_021\2006\12\28\page_021_2006-12-28_01-45
page_021\2006\12\28\page_021_2006-12-28_02-00
page_021\2006\12\28\page_021_2006-12-28_02-15

Each day adds about 12,200 files per web page (on average) and about 132 MB 
of used storage (on average).  Thus, there about 378,200 files per web page 
per month, utilizing about 4.092 GB of storage.  Space wise, the files 
should fit.  In reality, I can't make it work.  I can get about 9 to 10 
days of files for a single web page on a DVD at best, and that is if I'm 
willing to let Roxio run overnight to do the burning (it takes HOURS to 
burn those 10 days of data while the system just thrashes).

To give you some idea as to the thrashing that occurs, if, with nothing 
running on the PC, you right-click on the parent directory for a month for 
one web page, then select "properties" to get a count of the directories, 
files, and space allocated, you should plan on checking back in 40 minutes 
or so for the totals.

Oh, and the PC is a dual-core 3800 Athlon with 2 GB RAM and 2+ terabytes of 
disc space (used for professional photography).

I need to back up the files to DVD, but if anyone has some better ideas as 
to how to do it, please feel free to make suggestions.

Thanks!

John
*** When replying to this message, please do not delete these ***
*** signature lines. Otakon Katsucon HP3000-L @classiccmp.org ***
*** DigitalCosplay.com    JohnKorbPhoto.com     JohnPKorb.com ***

* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *

ATOM RSS1 RSS2