The Tufts High Performance Compute (HPC) cluster delivers 35,845,920 cpu hours and 59,427,840 gpu hours of free compute time per year to the user community.

Teraflops: 60+ (60+ trillion floating point operations per second) cpu: 4000 cores gpu: 6784 cores Interconnect: 40GB low latency ethernet

For additional information, please contact Research Technology Services at tts-research@tufts.edu


Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 267 Next »

UIT Research Computing Resources

For additional information, please contact Lionel Zupan, Associate Director for Research Computing, at x74933 or via email Lionel.Zupan@Tufts.edu.

Tufts UIT Research computing options

  • High-performance computing research cluster
  • Bioinformatics server
  • CarmaWeb server
  • Visualization Center
  • GIS Center

1. Tufts High-performance computing research cluster

What is a Cluster?

Cluster computing is the result of connecting many local computers (nodes) together via a high speed connection to provide a single shared resource. Its distributed processing system allows complex computations to run in parallel as the tasks are shared among the individual processors and memory. Applications that are capable of utilizing cluster systems break down the large computational tasks into smaller components that can run in serial or parallel across the cluster systems, enabling a dramatic improvement in the time required to process large problems and complex tasks.

Tufts Linux Research Cluster

The Tufts Linux Research Cluster is comprised of 40 identical IBM Linux systems (compute nodes) interconnected via an Infiniband network. Each cluster node has eight  2.8Ghz Intel Xeon CPUs and 16 or 32 gigabytes of memory for a total of 320 compute cores. The Linux operating system on each node is RedHat 5 configured identically across every machine. In addition there is a login node and a management node supporting the compute node array. Client/user workstations access the cluster via the Tufts Network or remotely with ssh. The user/login node has an additional network interface  that connects to the compute nodes using private non-routable IP addressing via the Infiniband hardware. This scheme allows the compute nodes to be a "virtualized" resource managed by the queueing software LSF, and abstracted away behind the user node. This approach also allows the cluster to scale to a large number of nodes and provides the structure for future growth.

The login node of the cluster is reserved for the use of compilers, running shell tools, and launching and submitting programs to compute nodes. The login node is not for running long running programs, etc... for computing purpose, please use the compute nodes and various queues.

Cluster User Accounts

Click Account Information for additional information about cluster accounts.

Contribute your own nodes to the new research cluster

Researchers that need their own high-performance computing (HPC) resources (and are applying for external grant funding to do so) may wish to consider contributing additional nodes to the research cluster rather than to develop and support their own HPC infrastructure. The research cluster has been designed to allow for this kind of compute node expansion. The obvious advantage to a researcher is that one does not have to support a separate computing resource, obtain additional licensing, etc.

In order to participate, additional nodes need to be of a certain kind, consistent with the current cluster design(as described above). In addition, a special LSF queue will be structured to allow one or more designated researchers priority access to the contributed cores. In return, when those cores are unused, they will become part of the larger pool of LSF managed compute node resources available to the Tufts research community.

For additional information, please contact Lionel Zupan, Associate Director for Research Computing, at x74933 or via email Lionel.Zupan@Tufts.edu.

Research Cluster Restrictions

Conditions and use of the research cluster include and are not limited to the following expectations. Additional related details may be found throughout this page.

Expectations

no user root access

supported OS is RedHat 5 Enterprise version

no user ability to reboot node(s)

all cluster login access is via the headnode

no user machine room access to cluster hardware

no alternative linux kernels other than that provided by RHEL 5

no access to Infiniband or Ethernet network hardware or software

no user cron or at access

no user servers/demons such as: HTTP, FTP. etc.

all user jobs destined for compute nodes are submitted via LSF's bsub command

all compute nodes follow one naming convention

only UIT NFS storage is supported

unused contributed node CPU time reverts to cluster user community

no user contributed direct connect storage

only limited outgoing Internet access from the headnode will be allowed; exceptions must be reviewed

allow 2-week turn around for software requests

Only user home directories are backed up

temporary public storage file systems have no quota and are subject to automated file deletions

Cluster quality of service is managed through LSF queues and priorities

Software request policy

Please send your request via email to cluster-support@tufts.edu and address the following questions:

  • What is the the name of the software?
  • Where can additional information about the software be found?
  • Who are the intended users of the software?
  • When is it needed by?
  • Will it be used in support of a grant and if so what grant?
  • What if any special requirements are needed?

Cluster Storage Options

Click here for details.

Network Concurrent Software Licenses

Click here

Support venue

If you have any questions about cluster related usage, applications, or assistance with software, please contact cluster-support@tufts.edu.

Cluster software environment

Click here

Installed Cluster Software

Click here

Compilers, Editors, etc...

Click here

Frequently Asked Questions - FAQs:

Cluster Connections/Logins

Click here

Parallel programming related information

Click here

Account related FAQs:

Click here

X based graphics FAQs

Click here

Application specific Information FAQs

How do I run program R interactively?

For interactive use type R-gui . The program will be dispatched on some compute node and any produced graphics will be displayed.

-bash-3.2$ R-gui

How do I run a long running R job as a batch job and I require lots of ram?

You may send your R script to the normal queue, as an example:

-bash-3.2$ bsub -q normal -R bigmem R CMD BATCH your-inputfile.r outputfilename

How do I run an MPI parallel program I compiled for use on 8 cpus?

For example, it the program requires interactive keyboard input, try:

-bash-3.2$ bsub -q express -Ip -n 8 mpirun.lsf ./yourprogram

How do I run Maple?

Maple has two interfaces. One is text based and the other is a graphical gui.

Text based:

-bash-3.2$ maple

Graphical:

-bash-3.2$ xmaple

How do I run gridMathematica?

How do I run StarP and Matlab?
-bash-3.2$ module load starp
-bash-3.2$ starp

StarP access is first come first served subject to load constraints and how many cores are requested. Note, StarP start up time delay on the order of 60+ seconds is normal. You may have to retry if you receive an error. This behavior is due to the LSF load balancing behavior during the job placement task. The default behavior of StarP is to use 8 cores. You may request up to 32 cores. For initial developement purposes please obtain proof of concept success with 8 cores before scaling.

For example, to use 16 cores:
-bash-3.2$ starp -p 16

IMPORTANT: Before running StarP, make sure that Matlab's Multithreading option is disabled. You can check this via Matlab's (File->Preferences->Multithreading) choice.

Where can I find StarP documentation?
You can find it on the cluster or the vendor's website. On the cluster it is located at /opt/starp/2.6.2/doc/Manuals/. Both html and pdfs are available. For html try:

-bash-3.2$ firefox file:///opt/starp/2.6.2/doc/Manuals/doc.html

Where can I find Chimera documentation?
If the gui and Help pull down option fails to open a browser then try:
-bash-3.2$ firefox file:///opt/chimera64/share/chimera/helpdir/index.html

How do I run StarP and Python?

How do I run a ggobi?

-bash-3.2$ module load ggobi

-bash-3.2$ ggobi

How do I run a Matlab?

-bash-3.2$ module load matlab

-bash-3.2$ matlab

  • How do I get matlab command line options?*
    -bash-3.2$ matlab -help

How do I run a matlab batch job?

-bash-3.2$ module load matlab

-bash-3.2$ bsub -q long "matlab -nojvm -nodisplay < your-matlab-code.m"

This will submit your code to the long queue. Matlab has several ways to direct output and results by use of commands such as: diary, print and other commands.

How do I speed up my Matlab calculations
By default matlab on the cluster has multithreading parallelism turned off. To enable it within Matlab choose: (File->Preferences->Multithreading). The alternative is to use StarP.

How do I run Fluent?

-bash-3.2$ module load fluent

-bash-3.2$ fluent

How do I run a Fluent parallel job?
Define the number of cpus(up to 8) within the bsub and fluent command line. The -n and -t arguments is the location to change the number of cpus.

-bash-3.2$ bsub -I -n8 -a fluent fluent 2d -g -t8 -pib -mpi=intel -lsf -i yourfluent.script
Note: replace 2d with the appropriate fluent product, such as 3d and so on...

How do I run Fluent's Gambit?

-bash-3.2$ module load fluent

-bash-3.2$ Gambit

How do I run Abaqus?
There are several ways to run jobs. Most will find the following useful:
-bash-3.2$ module load abaqus
-bash-3.2$ abaqus job=mydesign datacheck interactive input=./design.inp

This will run your abaqus input file called design.inp as a serial job and if successful will create about 14 files in the directory you are working in. All the files are created with the job name mydesign and differ in the filename extensions and type of file.

How do I run a parallel Abaqus job?
-bash-3.2$ abaqus job=design_4cpus cpus=4 mp_mode=mpi datacheck interactive input=./design.inp

Note different Abaqus products may invoke different solvers. Check the documentation for options.

How can I find the command line options?
-bash-3.2$ abaqus -help

How do I read Abaqus documentation?
If you have X11 forwarding in place you will get a webpage with documentation.

-bash-3.2$ module load abaqus
-bash-3.2$ abaqus doc

How do I run Mathematica?

-bash-3.2$ module load mathematica

For the gui interface:

-bash-3.2$ mathematica

For the text command line interface:

-bash-3.2$ math

Where can I find more Mathematica documentation?

The vendor's website and the online help in the application. In addition look at the broadcast section on the vendor site to view various movies on how to do many tasks in Mathematica.

My Mathematica notebook file doesn't work like it did before, what is up?
The following explanation is a good starting point to explore this issue.

How do I run Deform3D?

-bash-3.2$ module load deform

-bash-3.2$ deform3

Where do I find Deform documentation?

Both Html and Pdf manual versions are available at:
/opt/deform/3d/v61/MANUALS/

Is there a way to read the manuals on the cluster?

-bash-3.2$ firefox file:///opt/deform/3d/v61/MANUALS/HTML/DEFORM-3D.htm

How do I run Comsol?

-bash-3.2$ module load comsol

-bash-3.2$ comsol

How do I run Comsol as a parallel job?
Start Comsol as above, but add the command line options as follows:
-bash-3.2$ comsol -np 8 -mpmode owner

Then in the gui setup for Solver parameters, change the default Solver to the Parallel Direct Solver(Pardiso) and rerun your model.

Where do I find information on Imagemagick graphic tools?

Imagemagick home, usage info and scripting links. Also local man pages on individual commands is available.

Linux information FAQs

Where do I find basic unix/linux resources?

There are many web based tutorials and howto's for anything linux oriented. Some sites of interest:

linux-tutorial, Unix info , linux.org

What are some of the basic linux and related commands?

Most usage is centered around a dozen or so commands:

ls, more, less, cat, nano, pwd, cd, man, bsub, bkill, bjobs, ps, scp, ssh, cp, chmod, rm, mkdir, passwd, history, zip, unzip, tar, df, du
See the man pages for complete documentation. Here is a short description of some.

Basic Unix Commands

Action Needed

Command

Usage

Display contents of a file

cat

cat filename

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="bf5624c8-9988-4047-a8df-9e9954328dfa"><ac:plain-text-body><![CDATA[

Copy a file

cp

cp [-op] source destination

]]></ac:plain-text-body></ac:structured-macro>

Change file protection

chmod

chmod mode filename or
chmod mode directory_name

Change working directory

cd

cd pathname

Display file (/w pauses)

more

more filename

Display first page of text

less

less filename

Display help

man

man command or
man -k topic

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="4fdeebf3-c5b3-4162-bcb3-f7d46ce0ecc9"><ac:plain-text-body><![CDATA[

Rename a file

mv

mv [-op] filename1 filename2 or
]]></ac:plain-text-body></ac:structured-macro>
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="d2f91b10-992a-40e4-95dd-b7d233939f32"><ac:plain-text-body><![CDATA[mv [-op] directory1 directory2 or
]]></ac:plain-text-body></ac:structured-macro>
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="9282a262-3faa-46ea-ae00-e2826bed6421"><ac:plain-text-body><![CDATA[mv [-op] filename directory

]]></ac:plain-text-body></ac:structured-macro>

Compare file

diff

diff file1 file2

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="c667816f-9143-4a22-99aa-5ed4c6064156"><ac:plain-text-body><![CDATA[

Delete file

rm

rm [-op] filename

]]></ac:plain-text-body></ac:structured-macro>

Create a directory

mkdir

mkdir directory_name

Delete a directory /w files in it

rmdir -r

rm -r directory_name

Delete a directory

rmdir

rmdir directory_name

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="7d47034f-a77e-44aa-acd1-bf73cc54d2d8"><ac:plain-text-body><![CDATA[

Display a list of files

ls

ls [-op] directory_name

]]></ac:plain-text-body></ac:structured-macro>

Change the password

passwd

passwd

Display a long list (details)

ls -l

ls -l directory_name

Display current directory

pwd

pwd

What is a man page?

man pages are linux/unix style text based documentation. To obtain documentation on the command cat:

> man cat

>xman is the command for the x-based interface to man.

> man man is the man documentation.

> man -k graphics finds all related commands concerning graphics.

Are the compute nodes named differently from the old cluster compute nodes?

Yes. You should not hard code the names anywhere. The convention is node01, node02, ...

Some applications required a login to former node compute01, is this still the case?

No.

Why do I have to submit jobs to compute nodes?

The cluster has been configured to allocate work to compute nodes in a manner that provides efficient and fair use of resources. A job queueing system called LSF is provided as the work interface to the compute nodes. Your work is then distributed to queues that provide compute node resources. Login to compute nodes via ssh is not suggested and you will be asked to refrain from using the resouces in that manner; let LSF do it!

My program needs access to more than 16 gig or ram, what are my options?
An lsf resource has been defined to identify those nodes with 32 gig of ram. You access this through a bsub command line option, -R, when you submit your job.

-bash-3.2$ bsub -R bigmem -queue normal ./myprogram

Note: -bash-3.2$ is the default prompt for your bash shell on the cluster. The command is what follows it.

Are the LSF queues different than the old cluster:

No, for the most part. These may change in the future to accommodate needs.

How do I choose between queues:

You may view queue properties with the bqueues command:

-bash-3.2$ bqueues

And extra details by:

-bash-3.2$ bqueues -l |more

What are some of the most common LSF commands:

Action Needed

Command

Usage

System verification

lsid

lsid

Display load levels

lsmon

lsmon

Display hosts

lshosts

lshosts

Summarize past usage

bacct

bacct or
bacct job ID #

Display hosts

bhosts

bhosts

View current jobs

bjobs

bjobs or
bjobs job ID #

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="cc1990c7-5124-4ce0-ac2a-2347b7714c25"><ac:plain-text-body><![CDATA[

Run LSF batch job

bsub

bsub [-op] filename

]]></ac:plain-text-body></ac:structured-macro>

Kill a job

bkill

bkill job id #

Review/select queue

bqueues

bqueues or
bqueues queue_name

Suspend a job

bstop

bstop job ID #

Changes job order (new or pending)

btop

btop job ID | "job_ID"(index_list)"

Resume suspended jobs

bresume

bresume job ID #

View job history

bhist

bhist job ID #

How can I get notified when my lsf submitted jobs finish:

You need to add the -u option to bsub.
-bash-3.2$ bsub ... -u firstname.lastname@tufts.edu sleep 10

This will cause an e-mail to be sent when the job finishes, containing a summary of the job, the output, CPU & memory utilization, etc.

Compilation FAQs

How do I compile my old 32-bit code under the new 64-bit environment?

You should be able to recompile your code using the same compiler as you did on the old cluster. However, some compiler options may need to be 'tweaked'. For starters try using as few options as possible. For example, remove optimizations, check and debugging options. Investigate possible data alignment issues. If the code worked before, you shouldn't be too far from a solution. For new public domain codes, the task can be a much larger effort.

Where can I find some C++ info?
There are many internet resources. Try the compiler vendors first. Otherwise try:
C++

What recourse do I have if my old 32bit codes can't run on the cluster?
Contact us for access to a legacy 32bit linux node with gcc compilers.

Miscellaneous FAQs

How busy is the cluster:
One way to get a sense of this is from the Ganglia link.

What email and web services are available on the cluster:

The cluster does not accept incoming mail, nor is a webserver available for public use. These services are provided elsewhere by Tufts.

What is the backup policy:

Your data residing in your home directory is automatically backed up by UIT. A moving window of one year is used for backup purposes. It is possible to retrieve files as old as one year. For files less than one month old, these may be restored almost immediately. The policy adheres to industry standard best backup practices. To request a restore of data, contact the UIT support center x73376. You should have available basic info such as the file name(s) and approximately when it existed and what directory.

Is SPSS or SAS available on the cluster:

Neither is available on the cluster. Software packages R and Stata provide that functionality instead.

Where can I find information about PC/Mac based Tufts software licenses such as SAS or SPSS:

Here

Can I connect to the license server from home via my ISP to use Matlab on my Tufts laptop:

Programs such as Matlab and others that check out FlexLM based network concurrent licenses can not be used directly over the Internet, as you can while on campus. IP filtering limits license check-outs to the Tufts network domain. You may use the Tufts VPN solution to obtain check-outs.

Cluster user use cases (please click on link)

2. Bioinformatics services

a. Emboss and wEmboss:

Access to Emboss software is available on server http//emboss.uit.tufts.edu, which provides both shell and web access. In both cases you will need an account. The server hardware is a single quad core 64 bit host with 4 gig of ram.

For shell access to command line tools:
> ssh -Y emboss.uit.tufts.edu

For access to the web interface wEmboss.

For access to emboss web documentation.

Former GCG/Seqweb users can find equivalent Emboss functionality
here:

Emboss tutorial

If you have any questions about Emboss related usage, applications, or assistance with software, please contact bio-support@tufts.edu.

Bioinformatic related FAQs

Where are my old seqweb sequences:
Your old seqweb data is at: /nfshome/seqweb/users/your-user-name/

There you will find three directories with your data:
result state work

You may retrieve these with a file transfer program like
WinScp (http://(www.winscp.org) and store locally on your pc/mac. You may then
use a local web browser to look at the old seqweb data. You may also cut and paste sequence data into a wEmboss web session.

If I use the web interface to emboss where is data stored:
wemboss data is written into a directory called wProjects under your shell account. The path will be: /home/your-user-name/wProjects/

Will I have access to my old gcg shell account and data:
Your home directory on the old bioinformatic server is mounted as your directory on the new emboss server. However, access is via a shell login, not with the web interface, wEmboss.

b. Carmaweb server (please click on link)


UIT and the Medical School hosts and supports a web based service known as CarmaWeb. The focus of CarmaWeb is genetic microarray analysis. CARMAweb is a web based tool that allows the analysis of Affymetrix GeneChip, ABI microarrays and two color microarrays. The analysis includes normalization and data preprocessing, detection for differentially expressed genes, cluster analysis and GO analysis. These tools are built upon bioConductor and R software. One may request an account via the website.
Additional information here.

A CARMAweb tutorial is available here

The server hardware is a single quad core 64 bit host with 4 gig of ram.

If you have any questions about CarmaWeb related usage, applications, or assistance with software, please contact bio-support@tufts.edu.

3. Tufts Center for Scientific Visualization (or VisWall)


A description may be found here. The user guide is available here.

The research cluster is available to VisWall users for additional computational resources. Current connectivity follows standard practices using ssh and x11 forwarding. Viswall users with a cluster account may forward cluster based application graphic output for display on the VisWall. Future plans to integrate high speed network connectivity between the VisWall and research cluster is in development.

Monthly training classing on the use of the facility can be checked here

4. GIS Center


Several GIS links can be found here.

Tufts Research Cluster indirectly supports GIS spatial statistical computation with the availability of modern spatial statistics programs as found in R. This is a useful resource when faced with either complex estimation tasks, long runtimes or access to more memory than is often available on desktop workstations. R programs such as the following are available:

fields, ramps, spatial, geoR, geoRglm, RandomFields, sp, spatialCovariance, spatialkernel, spatstat, spBayes, splancs,

For additional information please contact cluster-support@tufts.edu.

5. Tufts ICPSR data subscription

The Inter-university Consortium for Political and Social Research (ICPSR) is a unit of the Institute for Social Research at the University of Michigan. ICPSR was established in 1962 to serve social scientists around the world by providing a central repository and dissemination service for computer-readable social science data, training facilities in basic and advanced techniques of quantitative social analysis, and resources that facilitate the use of advanced computer technology by social scientists.

The Tufts community may obtain research data and related web services from the ICPSR while one's computer is in the Tufts network domain. This is required for license authenication purposes. Special case exceptions are possible, but need to be arranged ahead of time. For additional information please contact cluster-support@tufts.edu.

  • No labels