.75 TB Wide-Area Network - Aware Cache now available for general useContact Brian Tierney (bltierney@lbl.gov) for access to dpss.lbl.gov
DPSS installation instructions are now located at: http://www-didc.lbl.gov/DPSS/admin/admin.html.
Anyone who needs large temporary data cache space may want to consider using
the LBNL DPSS (Distributed Parallel Storage System). The DPSS can provide
high-speed parallel I/O to any well connected hosts on the network.To use the DPSS, one can use the programs dpss_get/dpss_put to copy files
to/from the DPSS to their local system. Or, one can replace the UNIX I/O calls
in their program with DPSS I/O calls, and access the DPSS directly.Supported Platforms: Linux and Solaris (client library written in C)
Some caveats:
- DPSS is optimized for large read/write (> 1 MB at a time). If your application reads/writes data in small blocks, the DPSS is probably not the right answer
- To take advantage of the DPSS parallel I/O, your the end-to-end path from your host to the DPSS should be at least 50 Mbits/sec or greater. To find the speed of each hop, try using the program " pchar".
- The DPSS is not backed up, and is operated as a cache. The next user of the DPSS may flush your data from the cache. Also, there is currently no file level security. Anyone can read or remove your file. Security and cache reservations will be supported in the next release.
Important DPSS Utilities
Environment Variables:
- dpss_put
- dpss_get
- dpss_status: returns the current status and amount of free space of the DPSS
- dpss_ls (note: previously called dsm_list)
- dpss_remove
- dpss_rm
- DPSSHOST: hostname of DPSSMASTER to contact.
setenv DPSSHOST dpss.lbl.gov- DPSSTCPWINBUFSIZE: set the TCP socket buffers to this size. By default, the DPSS library tries to determine the optimal TCP buffer sizes for you. This takes about 1 seconds per connection. To eliminate this delay, set this variable. For info on determining the optimal buffer size, see the TCP tuning guide .
setenv DPSSTCPWINBUFSIZE 1024000NETLOGGERURL: send NetLogger monitoring data to this destination URL coming soon....
Troubleshooting:Try running dpss_status to check the DPSS:
setenv DPSSHOST dpss.lbl.govdpss_status
If the program dpss_status reports that the DPSS is down, you can restart it using the script:dpss_restart dpss.lbl.govSometimes when connecting to Linux servers, DPSS clients will report this error:
? [NoVer] : connect; Error<146> Connection refused ? [NoVer] : GetAvgRTDelayViaEcho; Error<146> Connection refusedIf you get this, set the DPSSTCPWINBUFSIZE env variable described above, to disable the GetAvgRTDelayViaEcho() routine.
If there is a problem with this page please, e-mail webmaster@www-itg.lbl.gov.