This is the draft page/content dump for the Tufts UIT Research Computing space. To be reorganized later.
UIT Research computing options:
- Linux research cluster
- Bioinformatic server
- CarmaWeb server
- Visualization Center
- GIS Center
- Cluster attached Database node
What is a Cluster?
Cluster computing is the result of connecting many local computers (nodes) together via a high speed connection to provide a single shared resource. Its distributed processing system allows complex computations to run in parallel as the tasks are shared among the individual processors and memory. Applications that are capable of utilizing cluster systems break down the large computational tasks into smaller components that can run in serial or parallel across the cluster systems, enabling a dramatic improvement in the time required to process large problems and complex tasks.
Tufts Linux Research Cluster
The Tufts Linux Research Cluster is comprised of 40 identical IBM Linux systems (compute nodes) interconnected via an Infiniband network. Each cluster node has eight 2.8Ghz Intel Xeon CPUs and 16 or 32 gigabytes of memory for a total of 320 compute cores. The Linux operating system on each node is RedHat 5 configured identically across every machine. In addition there is a login node and a management node supporting the compute node array. Client/user workstations access the cluster via the Tufts 188.8.131.52 LAN or remotely with ssh. The user/login node has an additional network interface that connects to the compute nodes using private non-routable IP addressing via the Infiniband hardware . This scheme allows the compute nodes to be a "virtualized" resource managed by LSF, and abstracted away behind the user node. This approach also allows the cluster to scale to a large number of nodes and provides the structure for future growth.
The login node of the cluster is reserved for the use of compilers, running shell tools, and launching and submitting programs to compute nodes. The login node is not for running long running programs, etc... please use the compute nodes and queues.
Access to Emboss software is available on server emboss.uit.tufts.edu, which provides both shell and web access. In both cases you will need an account.
For shell access to command line tools:
> ssh -Y emboss.uit.tufts.edu
For access to the web interface wEmboss.
For access to emboss web documentation.
UIT and the Medical School hosts and supports a web based service known as CarmaWeb. The focus of CarmaWeb is genetic microarray analysis. These tools are built upon bioConductor and R software. One may request an account via the website. Additional information here.
Research database(HPCdb) node
Cluster users may request access to mySQL database for supporting their research computing needs. Requests are treated like software requests. Please reference the Software Request Policy statement in this document.
Tufts Visualization Center
The research cluster is available to VisWall users for additional computational resources. Current connectivity follows standard practices using ssh and x11 forwarding. Viswall users with a cluster account my forward cluster based application graphic output for display on the VisWall. Future plans to integrate high speed network connectivity between the VisWall and research cluster is in developement.
Several GIS links can be found here.
Tufts Research Cluster supports indirectly, GIS spatial statistical computation with the availability of modern spatial statistics programs as found in R. This is a useful resource when faced with either complex estimation tasks, long runtimes or access to more memory than is often available on desktop workstations. R programs such as the following are available:
fields, ramps, spatial, geoR, geoRglm, RandomFields, sp, spatialCovariance, spatialkernel, spatstat, spBayes, splancs,
For additional information contact via email email@example.com.
Policy oriented topics:
UIT support center provides account management assistance. Access to the cluster requires a Research Cluster account. Faculty, other staff, and students are required to fill out a UIT Research Computer Account Application form, available at the following locations:
- Medford campus:
Contact the University IT Support Center at x7-3376 or by email at firstname.lastname@example.org for further assistance.
- Boston campus:
Present your Tufts ID during normal business hours at the Health Sciences Library 5th floor Learning Resources Center Helpdesk.
- Grafton campus:
Present your Tufts ID during normal business hours at the Veterinary Library during normal business hours.
Note: Students must have their application form signed by a faculty member or advisor.
You can also download a copy of the UIT Research Computer Account Application:
Click here to download a copy in Microsoft Word fomat
Click here to download a copy in Adobe PDF fomat
Grant Funding related topics:
Researchers that require deadicted access to compute node resources and are seeking funding for those resouces may wish to consider contributing additional nodes to the cluster. Tufts' research cluster has been designed to allow for compute node expansion. The obvious advantage to a researcher is that one does not have to support a separate computing resource, obtain additional licensing, etc. In order to participate, additional nodes need to be of a certain kind, consistent with the current cluster design. In addition, a special LSF queue will be structured to allow one or more designated researchers priority access to the contributed nodes. In return, when those nodes are unused, they will become part of the larger pool of LSF managed compute node resources available to the Tufts research community. For additional information contact Lionel Zupan, Associate Dir. for Reserach Computing at x74933 or via email: Lionel.Zupan@Tufts.edu.
Software request policy:
Please send your request via email to email@example.com and address the following questions:
- What is the the name of the software?
- Where can additional information about the software be found?
- Who are the intended users of the software?
- When is it needed by?
- Will it be used in support of a grant and if so what grant?
- What if any special requirements are needed?
Cluster Storage Options:
All accounts are created with fixed 200 megabyte home directory diskquota. In addition, a directory is automatically created in filesystem /scratch on the headnode and each compute node. The directory is named with your Tufts UTLN; such as /scratch/utln/. There is no quota and no backup for files located there. Typically ~100+ gig is available. In addition this filesystem is subject to automated cleaning based on a moving window of 28 days. You may use this for temporary storage supporting your research computing needs. Note, this storage is not available to your desktop as a mounted filesystem. Your access is via your ssh login or through a file transfer progam such as WinScp. For additional temporary storage beyond what is offered in /scratch, UIT provides a 2TB filesystem called /cluster/shared/. Access to this storage is by request and with suitable application requirement/justification. The naming convention is the same as /scratch/ ; /cluster/shared/utln/. Please work out of your named directory.
One additional filesystem is /tmp. This filesystem is used by applications and the operating system for temporary storage. Please note that it is under the same 28 day cleaning rule as /scratch.
Research Storage Solution:
Tufts UIT provides a network stroage based solution to faculty researchers. This solution is often used in support of grant based projects where the the amount of data, multiple access needs and backup are best meet as a network storage solution. A common mode of use it to support a small lab where multiple persons may interact with research data from several computers. Another possibilility is to request that the storage be made available to one or more accounts on the research cluster. For additional information contact Lionel Zupan, Associate Dir. for Reserach Computing at x74933 or via email: Lionel.Zupan@Tufts.edu. Click here to request Storage.
Network Concurrent Software Licenses:
Software on the research cluster is supported by a triple redundant FlexLM license server. This makes possible uninterrupted 24x7 license service requests by software clients. In addition, most of the licensed software on the cluster is "shared" with Tufts. In effect, access by Tufts owned workstations is available via various computer labs and faculty machines. Authenticated access is restricted within the Tufts network domain. Setup and additional information can be found here.
If you have any questions about cluster related usage, applications, or assistance with software, contact: firstname.lastname@example.org.
Cluster user software environment:
Each cluster shell account has a default bash shell upon login. This should meet the needs of 99% of most users. To change to other shells, such as csh, zsh, tcsh, use the chsh command. Additional info on the chsh command is available through the man pages.
Module management for software package user environment:
Software environments are managed through the use of the module approach. Many commercial packges and some public domain software require various settings that can often lead to clashes in the user shell environment. In order to use a package on the cluster, you must load that package's module. For example to use matlab:
> module load matlab
To see what packages are under module control:
> module avail
To unload a package from your envrionment:
> module unload matlab
To see loaded packages in your envrionment:
> module list
Currently there are some packages that are not under module control; such as DEFORM-3d, MaterialsStudio and maybe others.
Installed 64bit Cluster Software:
Platform Computing, Inc.'s LSF (Load Sharing Facility) software is a distributed load sharing and batch queuing suite of applications that can dispatch user requests to compute nodes in accordance with a Tufts-defined policy. It manages, monitors, and analyzes resources and load on the cluster. Platform LSF is layered in a way that allows it to sits on top of and extend the operating system services, speaking to the competing needs of resource management on the cluster. LSF commands must be used to submit batch jobs and assign interactive jobs to processors. bsub and lsrun are the usual command tools for this. It's important to note that cluster compute nodes are the only targets under LSF control. Jobs are not submitted to computers outside of the cluster. For more information about LSF command usage and job submission, you can read the man pages (example: type man lsrun at the cluster prompt). Most commands begin with the letter 'b' such as: bsub,bkill, lsrun, and bjobs.
Ansys is a suite of finite element based applications that provide real-world simulations of structural, thermal, electromagnetic and fluid-flow behavior of 3-D product. All Ansys products integrate with CAD environments.
Abaqus is a suite of applications used by many in the engineering community for the analysis of multi-body dynamics problems that aid the medical, automotive, aerospace, defense, and manufacturing community.
Fluent is a Computational Fluid Dynamics (CFD) software package commonly used in engineering education for research in fluid mechanics. The Fluent University Program provides universities with special, low-cost access to many of Fluents full-featured general use products. Each package includes a preprocessor, solver, and postprocessor.
MATLAB is a high-level technical computing language and interactive environment for algorithm development, data visualization, data analysis, and numerical computation. Using MATLAB, you can solve technical computing problems faster than with traditional programming languages, such as C, C++, and Fortran. Extensive documentation and tutorials are provided within Matlab. The following Matlab toolboxes are licensed:
Control System Toolbox
Distributed Computing Toolbox
Fuzzy Logic Toolbox
Image Processing Toolbox
Neural Network Toolbox
Partial Differential Equation Toolbox
Signal Processing Toolbox
Simulink Control Design
System Identification Toolbox
Virtual Reality Toolbox
Comsol is specifically designed to easily couple transport phenomena, including computational fluid dynamics (CFD) as well as mass and energy transport to chemical-reaction kinetics and process-related modeling. Licensed Modules include:
MultiPhysics, Chemical Engineering, Acoustics, Structural Mechanics, Script
Weka is a collection of machine learning algorithms for data mining tasks. The algorithms can either be applied directly to a dataset or called from your own Java code. Weka contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. It is also well-suited for developing new machine learning schemes.
Stata is an integrated statistical package for Windows, Macintosh, and Unix platforms. More than just a statistical package, Stata is also a full data-management system with complete statistical and graphical capabilities. It features both X-window and text user interfaces.
DEFORM (Design Environment for FORMing) is an engineering software environment that enables designers to analyze metal forming processes. DEFORM-3D is also a simulation system that is designed to analyze the three-dimensional flow of complex metal forming processes, allowing for a more complex analysis of shapes than 2D models can provide.
R is a widely available object oriented statistical package. The current list of installed packages can be found in directory /usr/lib/R/library/. This represents a base installation suitable for most routine tasks, however not all available packages as found on the web site are installed. If some other R package is needed, please make a software installation request as outlined above. Extensive user documentation and tutorials are available on the web site.
Materials Studio® is a validated software environment that brings the world's most advanced materials simulation and informatics technology.
It dramatically enhances your ability to mine, analyze, present, and communicate data and information relating to chemicals and materials. Materials Studio's accurate prediction and representation of materials structure, properties, and inter-relationships provides valuable insight. The following Materials Studio products are available: CASTEP, DMol.
Dacapo(available Fall 2008)
Dacapo is a total energy program based on density functional theory. It uses a plane wave basis for the valence electronic states and describes the core-electron interactions with Vanderbilt ultrasoft pseudo-potentials.
Chimera is a highly extensible program for interactive visualization and analysis of molecular structures and related data, including density maps, supramolecular assemblies, sequence alignments, docking results, trajectories, and conformational ensembles. High-quality images and animations can be generated.
Maple is a well known environment for mathematical problem-solving, exploration, data visualization, and technical authoring. In may ways it is similar to Mathematica and Matlab.
Star-P software is a client-server parallel-computing platform that's been designed to work with multiple Very High Level Language (VHLL) client applications such as MATLAB®, Python, or R, and has built-in tools to expand VHLL computing capability through addition of libraries and hardware-based accelerators.
MCCE (Multi-Conformation Continuum Electrostatics) is a biophysics simulation program combining continuum electrostatics and molecular mechanics.
WPP is a parallel computer program for simulating time-dependent elastic and viscoelastic wave propagation, with some provisions for acoustic wave propagation. WPP solves the governing equations in displacement formulation using a node-based finite difference approach on a Cartesian grid. WPP implements substantial capabilities for 3-D seismic modeling,
Mathematica and gridMathematica
Mathematica, advertised as a â€œone-stop shopâ€� for technical work, â€œintegrates a numeric and symbolic computational engine, graphics system, programming language, documentation system, and advanced connectivity to other applicationsâ€�. Not only does this application have parallel functionality built into it from the ground up, but the HYPERLINK "http://documents.wolfram.com/applications/parallel/" wolfram.com web site has extensive documentation, including numerous detailed tutorials.
ImageMagick® is a software suite to create, edit, and compose bitmap images. It can read, convert and write images in a variety of formats (over 100) including DPX, EXR, GIF, JPEG, JPEG-2000, PDF, PhotoCD, PNG, Postscript, SVG, and TIFF. Use ImageMagick to translate, flip, mirror, rotate, scale, shear and transform images, adjust image colors, apply various special effects, or draw text, lines, polygons, ellipses and Bézier curves.
ParaView is a multi-platform visualization application designed to visualize large data sets.
Spice is a general-purpose electric circuit simulation program for nonlinear dc, nonlinear transient and linear ac analysis.
Installed python modules: matplotlib, numpy, Networkx, Biopython
Perl is a stable, cross platform programming language. Perl is extensible. There are over 500 third party modules available from the Comprehensive Perl Archive Network (CPAN).
Portland Group compilers are available for use on the cluster. They are not part of the default environment on the head node, but they can be accessed by use of the module command. Fortran, C and C++ compilers and development tools enable use of networked compute nodes of Intel x64 processor-based workstations and servers to tackle serious scientific computing applications. PGI compilers offer world-class performance and features including auto-parallelization for multi-core, OpenMP directive-based parallelization, and support for the PGI Unified Binary™ technology.
GCC(C, C++, Fortran) compilers
The cluster 64-bit login node requires Gnu GCC 64-bit compiler and as a result becomes the default native compiler. No Module setup is required.
Documentation is available at GCC online documentation or from the following man pages:
> man gcc
> man g77
Tufts licenses the Intel compilers for use on the cluster. Access is via the following two commands:
ifort - Intel fortran compiler
icc - Intel C compiler
Local documentation in HTHL format can be found at:
or via manpages depending on what Module is loaded:
> man icc
> man ifc
Fortran quick reference is available by typing
> man ifort
Text Editing tools:
A web browser is provided to allow viewing of locally installed software product documentation. Access to the internet is restricted.
Frequently Asked Questions - FAQs:
What are some reasons for using the cluster:
- access to MPI based parallel programs
- access to larger amounts of memory than 32bit computers offer
- access to the large public domain of scientific computing programs
- access to compilers
- assess to large amounts of storage
- access to batch processing for running numerous independent serial jobs
- access to 64bit versions of programs you may already have on your 32bit desktop
Parallel programming related information:
What is MPI:
MPI stands for Message Passing Interface. The goal of MPI is to develop a widely used standard for writing message-passing programs.
What installed programs provide a parallel solution:
The following provide MPI based solutions: Abaqus, Ansys, Fluent, gridMathematica, StarP/matlab, StarP/python, paraview, MaterialStudio, Dacapo
The following provide thread based parallelism: comsol, matlab
When does 64bit computing matter:
When there is a need for memory and storage beyond the 32bit barriers.
Is it possible to run linux 32bit executables on the cluster:
There is a good chance that it will succeed. But there might be other issues preventing it from running. Try it out...
Where can I find additional information about MPI:
What is a good web based tutorial for MPI:
Account related FAQs:
What is the name of the cluster:
What is the Tufts responsible use policy:
Find it here.
How do I login to the cluster:
Use your Tufts UTLN and LDAP password associated with your Tufts email.
What is a Tufts UTLN:
This is your Tufts username issued for purposes of Tufts email.
When I login I see the follow, -bash-3.2$ , what is it:
This is the default prompt for your bash shell. This is where you issue linux commands.
How to transfer files:
Any file transfer agent program supporting either scp or sftp protocol will work. There are many freeware choices. WinScp for WindowXP is very good.
What is my home directory:
This is where you have access to storage that allows you to read, write and delete files. The path to your home directory is: /cluster/home/xx/yy/utln
where xx is the first letter of your first name
where yy is the first letter of your last name
where utln is your Tufts issued username for purposes of email.
What is the diskquota on home directories:
All accounts are created with a 200 megabyte quota.
May I request a diskquota increase to my home directory:
Often the best solution is to use cluster specific temporary storage options in conjunction with your desktop. Programs such as WinScp allow you to drag and drop files between the cluster and your desktop. You may use /scratch/utln for additional short term storage or if you require 100's of gigabytes, consider requesting access to /cluster/shared/ storage.
How do I find out how much total diskspace I have used:
Login to your account and type the following:
-bash-3.2$ du -sk .
You may also use WinScp or other such tools to view file summaries.
How do I obtain access to additional temporary storage on filesystem /cluster/shared/ :
Please send your request via email to email@example.com.
Connecting to the cluster FAQs:
How to connect to the cluster with a PC:
UIT research servers require an ssh connection to a host providing shell access. Use of Cygin ssh, SecureCRT, Putty, etc... will work. Other forms of connection such as XDM, rsh, rlogin, telnet are not supported.
What connection protocols are supported on the cluster:
ssh, ssh2, sftp, scp,
How to connect to the cluster with a Mac:
Open an xterm window and use ssh.
> ssh -Y cluster.uit.tufts.edu
Provide your username and password.
How to connect to the cluster with a linux OS:
Open a local shell window or xterm and connect with:
> ssh -Y cluster.uit.tufts.edu
Provide your username and password.
Can I connect to the cluster from home or while I am traveling:
Yes, use an ssh based login solution such as ssh, SecureCrt, etc...
Do I need SecureCRT to connect to a host:
No, if you use Cygwin you do not need SecureCRT. If you choose to use it, you will likely need a windows based xserver such as Exceed or Xwin32 or similar if you expect to display graphics.
X based graphics FAQs:
What is an xServer:
This is a program that runs on your workstation/desktop OS that 'listens' for X-Window transmissions sent from the cluster and redisplays these on your workstation. These transmissions are generated by an application running on a host that you are connected to. For example, if you intend to use Ansys on the cluster, you need to display the Ansys gui interface locally on your desktop.
What xserver is needed for WinXP desktops:
Where do I get Cygwin:
A&S Cygwin installation documentation can be obtained here.
What Cygwin programs do I install:
Install base Cygwin, OpenSSH, and OpenGL at a minimun.
How do I connect to the cluster using Cygin:
Connect with ssh to the headnode of the cluster:
> ssh -Y -C firstname.lastname@example.org
How can I make sure Cygwin is working with the cluster:
To test the cygwin xserver, try a simple cluster-side X-window application:
A clock should appear on your desktop.
What xserver is needed for a Mac: ?
An xserver is provided when you install the Mac X developement tools.**?????? **
What xserver is needed for a desktop linux user:
Linux distributions come with Xwindows which provides xserver support.
Application specific Information FAQs:
How do I run program R interactively:
For interactive use type R-gui . The program will be dispatched on some compute node and any produced graphics will be displayed.
How do I run a long running R job as a batch job and I require lots of ram:
You may send your R script to the normal queue, as an example:
-bash-3.2$ bsub -q normal -R bigmem R CMD BATCH your-inputfile.r outputfilename
How do I run an MPI parallel program I compiled for use on 8 cpus:
For example, it the program requires interactive keyboard input, try:
-bash-3.2$ bsub -q express -Ip -n 8 mpirun.lsf ./yourprogram
How do I run Maple:
Maple has two interfaces. One is text based and the other is a graphical gui.
How do I run gridMathematica:
How do I run StarP and Matlab:
How do I run StarP and Python:
How do I run a Matlab:
-bash-3.2$ module load matlab
How do I run a matlab batch job:
-bash-3.2$ module load matlab
-bash-3.2$ bsub -q long "matlab -nojvm -nodisplay < your-matlab-code.m"
This will submit your code to the long queue. Matlab has several ways to direct output and results by use of commands such as: diary, print and others....
How do I run MaterialStudio:
How do I run Fluent:
-bash-3.2$ module load fluent
How do I run Fluent's Gambit:
-bash-3.2$ module load fluent
How do I run Abaqus:
How do I read Abaqus documentation:
-bash-3.2$ module load abaqus
-bash-3.2$ abaqus doc
How do I run Mathematica:
How do I run Comsol:
-bash-3.2$ module load comsol
Where do I find information on Imagemack graphic tools:
Linux information FAQs:
Where do I find basic unix/linux resources:
There are many web based tutorials and howto's for anything linux oriented. Some sites of interest:
What are some of the basic linux and related commands:
Most usage is centered around a dozen or so commands:
ls, more, less, cat, nano, pwd, cd, man, bsub, bkill, bjobs, ps, scp, ssh, cp, chmod, rm, mkdir, passwd, history, zip, unzip, tar, df, du
What is a man page:
man pages are linux/unix style text based documentation. To obtain documentation on the command cat:
> man cat
>xman is the command for the x-based inteface to man.
> man man is the man documentation.
> man -k graphics finds all related commands concerning graphics
What is the local storage automated cleaning policy:
The headnode of the cluster and compute nodes provides two areas of temporary storage separate from your home directory. File systems /scratch/utln/ and /tmp are available. Each is subject to automated cleaning rules. All files older than 28 days are deleted and these file systems are not backed up.
Are the compute nodes named differently from the old cluster compute nodes:
Yes. You should not hard code the names anywhere. The convention is node01, node02, ....
Some applications required a login to former node compute01; is this still the case:
Why do I have to submit jobs to compute nodes:
The cluster has been configured to allocate work to compute nodes in a manner that provides efficient and fair use of resources. A job queueing system called LSF is provided as the work interface to the compute nodes. Your work is then distributed to queues that provide compute node resources. Login to compute nodes via ssh is not suggested and you will be asked to refrain from using the resouces in that manner; let LSF do it!
My program needs access to more than 16 gig or ram, what are my options:
An lsf resource has been defined to identify those nodes with 32 gig of ram. You access this through a bsub command line option, -R, when you submit your job.
-bash-3.2$ bsub -R bigmem -queue normal ./myprogram
Are the LSF queues different than the old cluster:
No, for the most part. These may change in the future to accomondate needs.
How do I choose between queues:
You may view queue properties with the bqueues command:
And extra details by:
-bash-3.2$ bqueues -l | more
What are some of the most common LSF commands:
bsub, bkill, bjobs, bqueues, bhist
What email and web services are available on the cluster:
The cluster does not accept incoming mail, nor is a webserver available for public use. These services are provided elsewhere by Tufts.
What is the backup policy:
Your data residing in your home directory is automatically backed up by UIT. The policy adheres to industry standard best backup practices. To request a retrivial of data, contact the UIT support center x73376. You should have available basic info such as the file name(s) and approximately when it existed and what directory.
Is SPSS or SAS available on the cluster:
Neither is available on the cluster. Software packages R and Stata provide that functionality instead.
Where can I find information about PC/Mac based Tufts software licenses: