Server/NAS 2.0

Message

#1 Post by **palmboy5** » Thu Aug 12, 2010 1:14 am

So, now that my new server is mostly done, here's a new thread "documenting" it.

Specs:
FreeBSD 64bit 8.1-RELEASE
PC Power and Cooling PPCS370X 370W
AMD Athlon 64 X2 4000+
Gigabyte GA-MA69GM-S2H w/ BIOS version F6
Corsair XMS2 2x1GB DDR2-800
Western Digital AV-type 80GB IDE OS drive
2x Western Digital WD20EADS 2TB
2x Western Digital WD20EARS 2TB

It used to be 1x WD20EADS and 3x WD20EARS but one of the EARS died an early death. Then I read up on Western Digital's quality control and saw that they only weed out infant death drives for their higher end models. *sigh*

I switched from Ubuntu with mdadm doing RAID5 to FreeBSD with RAIDZ(RAID5) and ZFS. ZFS being 398473987x more superior than any other file system, ever (look it up!). What bugged me about RAID5 was the need for battery backup. Silent data corruption due to the RAID5 "write hole" was something I was not going to put up with

.

One learning curve later and a day's time worth of data transferring..
http://www.mylilsite.net/images/compute ... /6tb-2.png

My basic numbering
http://www.mylilsite.net/images/compute ... beled1.jpg
they correspond to the labels I gave the HDDs in software. You can see the damage to HDD #1 from when I pulled a locking SATA plug out without pressing down the locking tab.

I need a bigger case!
http://www.mylilsite.net/images/compute ... dRAID1.jpg
http://www.mylilsite.net/images/compute ... dRAID2.jpg
http://www.mylilsite.net/images/compute ... dRAID3.jpg
http://www.mylilsite.net/images/compute ... dRAID4.jpg

2005 · #2 Post by **2005** » Thu Aug 12, 2010 5:58 pm

I want a server..

#3 Post by **palmboy5** » Sun Aug 15, 2010 5:59 am

lol, porting the checksumming program I wrote for Linux to BSD is proving to be more of a bitch than I expected.

2005 · #4 Post by **2005** » Mon Aug 16, 2010 7:53 pm

checksumming program... do tell ?

#5 Post by **palmboy5** » Tue Aug 17, 2010 1:31 am

Two things I found out regarding doing SHA512 hashes..

1. Even the Atom CPU is bottlenecked by the HDD.
2. ExactFile in Windows on a 3.2GHz Q6600 is slower than sha512sum in Linux (haven't benchmarked shasum in FreeBSD) on a 1.6GHz Atom. Even with four files (threads) at once on the quad, it still wasn't faster than one thread on the Atom.

Conclusions: ExactFile sucks. Do my checksumming directly on the server.

Problem: There is no program that I know of that will do SHA512 in any way that allows me to let it go across all files in all directories automatically. Individual manually defined files, sure.. but not entire directory trees.

Soooo, I write my own

. This way I can also add features I really want but haven't seen any program have.. The ability to detect matching checksums and ask if the file was renamed. The ability to ask if a file that failed its checksum was modified by the user, as opposed to corruption. For both cases the program needs to be able to update the checksum in the digest file. Oh! Also the ability to update the digest file with checksums of new files.