Taking Home Your SHARC-2 Data

In principle, there are two ways you can hold on to your data, after you have spent so much time collecting it. The more convenient method is probably to take the data with you on a set of CD's or DVD's. Alternatively, you can simply transfer your data to your computer (laptop here, or desktop far away) via FTP, NFS or Windows Networking. The latter seems easy, but the size of your SHARC-2 data may give you a second thoughts. As a reminder, SHARC-2 will, on a good night, generate 2-3 GB of data!!! That's quite a lot even with decent network bandwidths.


The data Location.

We keep two copies of your data at all times for security. The copy you can (and should) access can be found on the champinux computer in 

    champinux:/home/transfer/sharcii/data_YYYYmmm

i.e. for the April 2004 run, that will be:

    champinux:/home/transfer/sharcii/data_2004apr

Normally, the directory of the current run is also linked to:

    champinux:/home/transfer/current

for easier access.

The directory champinux:/home/transfer is exported under NFS, and can be mounted as read-only from anywhere on the summit network. This means, that you should be able to mount it as an NSF disk from your laptop as well, if it is plugged into the network up there, or at the HP office (though transfer rates at HP are really not worth it!). Similarly, the current data directory champinux:/home/transfer/current is made available for Windows Network access (if you are wondering -- the reason not the entire transfer drive is shared is for security)

The data files follow the simple naming convention of sharc2-######.fits after their scan numbers. I.e., the scan 14282 will be written as sharc2-014282.fits in the appropriate directory on the champinux:/home/transfer drive.

If, for some reason, your data appears not to be present there, please contact someone at Caltech, and we will make sure to remedy the problem as soon as possible.


Accessing your data via NFS

Any computer on the CSO summit network is allowed to mount the data drive via NFS. That means that if you have a laptop with you, you can use NFS to transfer your data to it. To access the data drive from under Unix (Linux or MacOS X), first create a directory that will serve as the mount point for the transfer drive on your computer. E.g.:

    mkdir ~/sharc-data

Now you can mount the transfer drive into that directory with the command:

    mount -t nfs champinux:/home/transfer ~/sharc-data

Once mounted, you'll find your April 2004 data in ~/sharc-data/sharcii/data_2004apr (or, ~/sharc-data/current for the current observing run) on your computer. (Similarly for other runs -- the naming should be clear from the example). Now, you can copy the data files to their resting place on your own computer. Say you will want to keep them in ~/mydata (create the directory by mkdir ~/mydata first, if it does not exist yet), then

    cp ~/sharc-data/current/sharc2-01428*.fits ~/mydata

will copy the scans 14280-14289 into your ~/mydata directory.

When finished simply disconnect by typing

    umount ~/sharc-data

If you no longer need it, you may as well delete the mount point by typing:
    rmdir ~/sharc-data



Accessing your data from Windows

You can also access the SHARC-2 data for the current run from Windows Networking. To do that, follow the steps below:

    1. Open Network Neighbourhood.

    2. Double-click Entire Network.

    3. Select the workgroup Csosamba.

    4. Select the computer Champinux.

    5. Open the folder Current Data.



FTP Access

There is no conventional FTP access to the data due to security reasons. You can, however, try accessing via SFTP (SSH Secure File Transfer in Windows provides you a nice GUI too!). SFTP works just like FTP. To log in, type:
    
    sftp sharc@champinux.submm.caltech.edu
   
Alternatively, you can log onto champinux, or kilauea as sharc (you have probably done the latter already to operate the instrument...), and open an ftp connection out to your FTP server.



Write Data to DVD's or CD's

You may burn your data onto CD's or DVD's using the Sony DVD burner on the champinux computer (The big black Dell server, in the far corner of the smalller control room). You may find empty media and jewel cases on, or around, champinux too . Use them as needed, and please let someone at Caltech know if the stack is running low, so we may arrange replacement in due time. Of, course, you are always welcome to use your own media as well.

Choosing a medium.

Writing your data to DVD or CD is likely to be the most convenient way of taking your data with you. First, you should decide which medium you prefer. CD's hold up to 640MB (sometimes 700MB) of data, which will typically accomodate 2-3 hours of SHARC-2 data. DVD's have capacity for 4.6GB, and that should be enough for 1-2 full nights. Depending on how much data you have taken during your run, you may make the choice of the medium yourself.

Archive or append.

Next you should decide the method of writing. You can create both single and multi-session CD's and DVD's. The advantage of multi-session disks is that you can keep appending data to the disk until it's full, making more efficient, when writing discontiguous data sets. Multi-session disks are created by the append-night and append-scans command. The syntaxes are:

    append-night YYYYMMDD [hhmm hhmm]
    append-scans start# end#

When archiving large sets of data (e.g. several nights at once), so that they would not fit to a single disk, you can use one of the archive commands archive-scans or archive-night. The script archive-night will write and close a single disk with the given night's worth of data (Note. this means that you cannot add data to this disk at a later time). Then you may proceed to the next night on another disk until you are done.

On the other hand, the script archive-scans, will archive any range of scans to multiple disks if necessary, in a much more space efficient way. The script will prompt for the number of disks necessary, and then you may decide whether or not you want to go ahead (provided you have the required number of disks at hand). The syntaxes for the archive scripts are:

    archive-night YYYYMMDD [hhmm hhmm]
    archive-scans start# end#

Guide to options: 
  
YYYMMMDDD
 is UT date. E.g. 20040423 for 23 April 2004. 
[hhmm hhmm]
 Optionally specifies starting and ending times in UT. When the hours are not explicitly specified it is understood that the entire night will be written to the disk. 
start# end#
 Inclusive starting and ending scan numbers. Leading zeroes can be ommitted.
   
        
By scan numbers or by night/time.

If you are in dillemma whether to write disks by night or by scans, here's some help. Writing by night will write the entire night's worth of data by default. With the option of specifying starting and ending UT times as well, the *-night scripts are well adapted to writing half nights too. The advantage is primarily the fact that you don't have to look through your logs for actual scan numbers. On the other hand, archiving by scan numbers gives you more exact control if you will. Choose whichever suits you best...



Request CD's or DVD's from Caltech

If all fails, and you left the CSO without a hand on your data, there is a last resort emergency route. We, at Caltech, keep all data (with at least two copies at all times, for surviving media failures). So do not despair. You will always have your data, but you'll have to make arrangements with someone at Caltech to access it (we can burn and send you DVD's, or provide ftp access to your data -- whichever you prefer. Naturally, we will always prefer that you choose one of the other options described above...