
On this website you can find documentation about software commonly used in bioinformatic data analyses as well as tutorials about various bioinformatic subjects. On this webpage you can find software organized by topic and for each topic you find a list of commonly used software tools.
If you are working at the University of Amsterdam (UvA) Institute for Biodiversity and Ecosystem Dynamics (IBED) and want to know more about what computational resources are available, please also visit the computational support teams website and our website with more computation resources
Please, be aware that this page is a work in progress and will be slowly updated over time. If you want to add additional information or feel that something is missing feel free to send an email to n.dombrowski@uva.nl.
Useful tutorials
If you are new to informatics it is recommended to start with the bash and HPC tutorial below. It teaches you how to use the command line and how to run analyses on Crunchomics, the UvA computer cluster, and a lot of the other tutorials on this page are build on it.
Getting started with bash
- Getting started with bash and an HPC: Start here if you have never used the command line. You learn how to set up a terminal and how to organize and inspect sequencing data with bash. Then you copy the data to Crunchomics, install software and run your first jobs there. While this tutorial is build around sequencing data it applies to other project with large datasets or many files.
- Working with tables using AWK and sed: AWK and sed are command line tools for tables. You can use them to replace patterns in a file, pull out columns, keep only the rows that match a pattern, or merge two tables. To follow this tutorial you need the command line basics first.
- Keeping track of your code with git: Git saves a history of the changes to your scripts, so you can go back to an earlier version and share your code with others. This page gives a short overview of the most common git commands.
Writing bash scripts
Once you know the basics, these pages help you write scripts that run on many files, stop safely when something goes wrong, and can be submitted again. Read the first three in this order:
- For-loops in bash: How can I run the same tool on many files instead of one?
- Checking files in bash scripts: How can my script check whether an input file is there, and skip a sample or stop with a clear message when it is missing?
- Skip finished samples safely: How can I submit a job again so that it skips the samples that already finished, and only those? This page also lists what to look for in a script that a GenAI tool wrote for you.
Read these when you need them:
- Stopping scripts on errors with
set -euo pipefail: Why does AI often add this line to the top of a script, and when does it help or cause problems? Read this when a script you got has this line, or when your own script uses a pipe (|). - Bash arrays: How can I store a list of file names in one variable, and when do I not need to?
Storing data on Crunchomics
- Using the local scratch on Crunchomics: Each compute node on Crunchomics has its own fast, temporary storage. This page shows how to use it for jobs that read or write a lot of data. Read it once you are comfortable running jobs on Crunchomics.
- Using the IBED Tape Archive: Crunchomics is not meant for long-term storage. This page shows how to copy raw data and finished results to IBED’s tape archive.
Using R
- R cookbook: Have a look at this tutorial if you are new to R, it requires no coding experience. It takes you from installing R and RStudio to reading, changing, plotting and testing your own data. Work through the first chapters in order.
- Changing tables with the tidyverse: More practice with filtering, sorting, changing and summarizing tables with dplyr, using a data set of penguin measurements.
- Making figures with ggplot2: Build scatter plots, bar plots, boxplots and more, and adjust colors, axes and themes until a figure is ready for a report or paper.
Bioinformatic workflows
Pick the group that matches your data.
For all sequencing data
- Downloading data from NCBI: Download sequencing reads (from the SRA) and genomes from NCBI, for one or more samples.
- A first look at your sequencing data: Check your data soon after you receive it. You count the reads per sample and make quality reports with FastQC, so you can catch problems while they are still easy to fix. You can follow this tutorial without any command line experience.
Amplicon data
- From sequencing reads to an ASV table with QIIME 2: Use a 16S rRNA gene test data set to remove primers, turn reads into amplicon sequence variants (ASVs) with DADA2 or Deblur, assign taxonomy and calculate diversity. Includes a section on running QIIME 2 on Crunchomics.
- Analysing an OTU or ASV table in R: Filter, normalize and visualize your table in R, for example with alpha and beta diversity plots, bar plots of the taxa, and tests for taxa that differ between groups.
Microbial (meta)genomes
These tutorials were first written for workshops at NIOZ. Some use older tool versions or NIOZ file paths, so you may need to adapt the commands for Crunchomics. They also expect you to be comfortable with bash and AWK.
- Assembling a metagenome: Check and clean the reads of three small metagenomes, assemble them into contigs with MEGAHIT and metaSPAdes, and compare the assemblies.
- Recovering genomes from metagenomes (binning): Sort the contigs from an assembly into metagenome-assembled genomes (MAGs) with several binning tools. Then combine the results with DAS Tool and check genome quality (CheckM) and taxonomy (GTDB-Tk).
- Annotating microbial genomes: Predict the genes in a set of genomes and find out what they might do, by comparing them against databases such as COG, arCOG, KEGG and Pfam. All results end up in one table.
- Building a species tree: Learn how phylogenetic trees are built. Then align marker proteins, trim the alignment and build trees with IQ-TREE, for single proteins and for several proteins combined.
Custom scripts
- Custom scripts: A list of custom scripts useful for bioinformatic analyses
Useful databases A-Z
- arCOG: Archaeal Clusters of Orthologous Genes (arCOGs) database to analyse archaea
- COG: Cluster of Orthologous Genes (COGs) to analyse bacteria
- dbCAN: Database to analyse carbohydrate-active enzymes in genomes
- FeGenie: Database to identify iron genes and iron operons in genomes
- HydDB: Database to identify hydrogenases in genomes
- KOfam: Database to assign KEGG orthologous (KOs) to protein sequences
- NCBI-nr: NCBI’s non-redundant (nr) protein database to analyse protein sequences
- Pfam: A collection of protein models to analyse protein sequences
- PGAP database: Successor to the TIGRFAM database
- TIGRFAM: A collection of manually curated protein families to analyse prokaryotic sequences
Bioinformatic tools A-Z
A-D
- ATLAS: A metagenomic pipeline for QC, assembly binning and annotation
- Augustus: A program that predicts genes in eukaryotic genomic sequences
- Autocycler: A tool for generating consensus long-read assemblies for microbial genomes
- Bakta: A tool for the rapid & standardized annotation of bacterial genomes and plasmids from both isolates and MAGs
- Barrnap: A tool to predict the location of ribosomal RNA genes in genomes.
- Binette: A tool to generate a non-redundant bin set
- BMGE: A program to select regions in a multiple sequence alignment that are suited for phylogenetic inference
- Bowtie2: A tool for aligning sequencing reads to genomes and other reference sequences
- BUSCO: Quality assessment of (meta)genomes, transcriptomes and proteomes
- BWA: A tool to map DNA sequences against a large reference genome
- CheckM2: A tool to assess the quality of a genome assembly
- Chopper: A tool for quality filtering of long read data
- COMEBin: A tool for reconstructing genomes from metagenomic data
- CoverM: A DNA read coverage and relative abundance calculator focused on metagenomics applications
- DAS Tool: A tool to generate a non-redundant bin set
- DeepLoc: A tool to predict the subcellular localization(s) of eukaryotic proteins
- DeSeq2: Analyse gene expression data in R
- Diamond: A sequence aligner for protein and translated DNA searches
- Dnaapler: A tool to re-orient a genome, for example at dnaA
E-H
- FAMA: A fast pipeline for functional and taxonomic analysis of metagenomic sequences
- FastP: A tool for fast all-in-one preprocessing of FastQ files
- fastplong: A tool for ultrafast preprocessing and quality control for long reads
- FastQC: A quality control tool for read sequencing data
- FeatureCounts: A read summarization program that counts mapped reads for genomic features
- FeGenie: A HMM-based identification and categorization of iron genes
- Filtlong: A tool for filtering long reads
- Flye: A de novo assembler for single-molecule sequencing reads
- GTDB_tk: A software toolkit for assigning objective taxonomic classifications to bacterial and archaeal genomes
- GToTree: a user-friendly workflow for phylogenomics intended to create phylogenomic trees
- HMMER: A tool for searching sequence databases for sequence homologs, and for making sequence alignments
- Homopolish: A tool for the removal of systematic errors in nanopore sequencing by homologous polishing
I-L
- Interproscan: A tool to scan protein and nucleic sequences against InterPro signatures
- IQ-TREE: A tool for phylogenomic inferences
- ITSx: A tool to extract ITS1 and ITS2 subregions from ITS sequences
- Kraken2: A taxonomic sequence classifier using kmers
M-P
- Mafft: A multiple sequence alignment program
- Medaka: A tool for assembly polishing
- MEGAHIT: A de novo assembler for short reads, often used for metagenomes
- MetaBAT2: A tool for reconstructing genomes from metagenomic data
- METABOLIC: A tool to predict functional trait profiles in genome datasets
- MetaCerberus: A tool for functional assignment
- Minimap2: A program to align DNA or mRNA sequences against a reference database
- MOTUS: A tool to estimate microbial abundances in Illumina and Nanopore sequencing data
- MultiQC: A program to summarize analysis reports
- NanoClass2: A taxonomic meta-classifier for long-read 16S/18S rRNA gene sequencing data
- NanoITS: A taxonomic meta-classifier for long-read ITS operon sequencing data
- Nanophase: A pipeline to generate MAGs using Nanopore long and Illumina short reads from metagenomes
- NanoPlot: Plotting tool for long read sequencing data
- NanoQC: A quality control tool for long read sequencing data
- NGSpeciesID: A tool for clustering and consensus forming of long-read amplicon sequencing data
- Porechop: A tool for finding and removing adapters from Nanopore reads
- Prokka: A tool to annotate bacterial, archaeal and viral genomes
- Pseudofinder: A tool that detects pseudogene candidates from annotated genbank files of bacterial and archaeal genomes
- pyCirclize: A tool for circular visualization, i.e. genome plots, in python
- pyGenomeTracks: A tool to produce high-quality genome browser tracks
Q-Z
- QUAST: A Quality Assessment Tool for Genome Assemblies
- Ribodetector: Detect and remove rRNA sequences from metagenomic, metatranscriptomic, and ncRNA sequencing data
- RSEM: A software package for estimating gene and isoform expression levels from RNA-Seq data
- RSeQC: A tool to evaluate high throughput sequence data especially RNA-seq data
- Samtools: A tool to manipulating alignments in SAM/BAM format
- SemiBin2: A tool for reconstructing genomes from metagenomic data
- SeqKit: A tool for FASTA/Q file manipulation
- SignalP6: A tool to predict the presence of signal peptides
- SingleM: A tool for taxonomic profiling of shotgun metagenomes
- SortMerNa: A tool to filter ribosomal RNAs in metatranscriptomic data
- STAR: An ultrafast universal RNA-seq aligner
- StringTie: A a fast and highly efficient assembler of RNA-Seq alignments into potential transcripts
- TD2: A tool to identify candidate coding regions within transcript sequences
- TransDecoder: A tool to identify candidate coding regions within transcript sequences
- Trinity: A tool to assemble transcript sequences from Illumina RNA-Seq data
- Trycycler: A tool for generating consensus long-read assemblies for bacterial genomes
- VAMB: A tool for reconstructing genomes from metagenomic data