BED, GFF, GTF, and GFF3 formats

GFF, GTF, GFF3 & BED files are all file formats that are used to store annotation (features) generally without containing any sequence. Although it is common that they will be accompanied by a fasta file containing the sequence only. They emerged as a way of exporting, or exchanging, information from a specified region of an entire genome without having to take the entire genome.

Most sequence formats were developed to be for a specific gene or protein. Although this is no longer true they are still orientated to be of a region of fixed length. These annotation files are not at all length specific and could potentially store just two features that were at either end of the same chromosome. They are a much more flexible way of dealing with annotation, especially a large amount, than a fixed length sequence format such as Genbank.

They also are not limited to a single sequence and can contain information from multiple sequences in the same file (Fasta files can also contain multiple sequences). For example you could store the entire human set of chromosomes in a pair of (quite large!) files. A multiple sequence Fasta file and a single GFF file.

The format of these annotation files does vary (who ever said Bioinformaticians had to be consistent!) but basically their format consists of a set of individual lines (one line per feature) along the following lines:

SEQUENCE ID, START, STOP, FEATURE TYPE, NOTE

Sequence ID is the sequence these annotations belong to.

START and STOP are the region of sequence they are annotated against

FEATURE TYPE is obvious! Note that this does not always correspond to a correct Genbank Feature Keyword

Genome Browsers

These tools (generally online web gateways) allow you to browse the entire chromosome or genome of a particular organism. Almost like a graphical model of a sequence database. All the information known about that particular organism’s sequence that has been submitted to one of the large sequence databases (e.g. Genbank at the NCBI) should be visualised within the genome browser. You can download all the annotation contained within a particular region fairly easily using one of these annotation formats. Then you can either annotate an existing file that you are working with (so preserving your own “private” annotation with all known public annotation).

To annotate a sequence with a BED/GFF/GFF3/GFT file in MacVector

From the UCSC’s Genome browser

The interface will change and show all annotation associated with that region. You can modify the amount or type of annotation being showed. This particular gene, C.elegans Sel-12 is located on Chromosome X

This will now allow you to export all the annotation associated with the previous displayed region (tracks).

If it is left at genome the entire genome will be downloaded

Now you need to open the sequence you want to annotate. For this example we could go to DATABASE > ENTREZ and search for and download Accession Number U35660. However, that only contains the mRNA and not the genomic sequence. So instead download the fasta sequence from the NCBI

Sequence : Chromosome: X; NC_003284.7 (915873..918235)

You will need to ensure that the start position of the downloaded file corresponds to the start of the region of the chromosome we have just downloaded the annotation for. This is easily done in MacVector.

Now we will import our downloaded annotation.

The Sequence ID (SeqID) contained within all features in the file will be shown in a dialog along with the number of features for each SeqID and the region of the sequence that these will be annotated against. A warning will be displayed if any of the features are outside of the region to be annotated.

A dialogue will be shown with the number of annotations that have been added. If you annotate a blank sequence (e.g. a fasta file) the resulting features may be initially hidden. However, you can easily show them from the Graphics Palette tree view.

You can choose to annotate your sequence with all the features contained within the imported file or to ignore duplicates.

ToxoDB Genome browser

Here’s a similar workflow from the ToxoDB Genome browser

Toxoplasma gondii ME49

"CCTTCCCTGCGTCAGAGGAGAAGAGAACGGCTTGACCGATGGAGGACCCCGCAAACATGAGGGCGAAGGTAGTCTGCATGATCTCTGAACAAGGAACACGGCGCGGAAAGGGAAGCACAGAAGGAAGTCGATCAAGACACCCTGCGTTGTTTTTCGGGGAGCCCCAGAGAGGGAGCTCGCGGCTCTGGACTTCAAGGTCCGTCGAGCAGCAGAACGCTTCACTCGGCAAGGAAGGAGCAGTTTCTTCTCTCGCGTCTTGTTTCGCTTTCACGGCTTCGTTTTCTCGCCGCGACCTGCGAAAAGAAAACAGCTCCCCTATAAGAACTCGACTCTCGAGCCTGCGGTTTGGTATCGGCTTTTTCTTCAGAGTTTTTTCTGTCGCGCGTTCGGACAACCAGTTCCGTGCTTGCGCGCCTCCTCTGAAGGCCGCGCCCGCCTCTCGACTCCCGTCGCTTCTCTCTTCGGCTGGATAAGAGAAAACGCTGAACGAAGAGGAGAGTACGCACTGGCATCGTTTGTCGACTTTCGTCTCCAGGTGGGGGAGTGTCGGTCGACTCACCCAAGGGATTCTTCCCTTCGCTGCTCACGATCTGGCCGCCATACCAAAAAATCAGCGCCTGCAGAGCGTACTGAGCTCCCTGACAGACACGCAGACGCAGCGGCAAGGAGACGCTGAAAAGAAGAAAGACAACCGGAGAGCGCGGAGAACAAAGAACTGTGAGCGTGCAACGACGGGATCAAGGACGACAGCGAATCTCCCGTCTTCAGGACCTCGACGGGCATTCCGCATGGCAGGTCCTTTGACTCCGAAAAACTCTGCGGCAGCCTCGATGACCCTTACCCCCCCGGAACATCCCCGAGAGCTCGGTGGAAAAAACCTCTCATTCGAGAGCGACAGATCAGGCTTTGCTAGTCGAGCCAGAAGGCAGGAGAAGGAACGGAGCGAACCGCGGATGCGTCTCTCTGCGCGGACGAGTCTTCATGAGCAGGCACCGCGACGTTCCAAGAAGCAGAAAGAGAGAGAGGAGAGAGGAGAGAAGCGAGAAGCTCGGGAACTCACGAGGAAGCAACAAAGATCTCTCCTCGTCACTCACGTTCCGTCGACCTGCATGGCAGGCGTGACGCGGCATGCACAGCAGAAGACCTTTCGAGGTCACCACACACACCGCCTCGGACGTCGAGAAGTCTCGATCTTTGTGAACCACAGGGCTCTGTTTTGTGTGGCGGAACGAAGAAACCAAGCGCTTAGGATGGAGCTCACTGGAGAGCAGGAAACGGATCTTCAAACGAGTGTCGACGTCCTCCCGCGCATCCGAACCGAAACTCAAACGCGCTCCAGAGAGACGACATAGAAGACAGAGACGTACAATGAGAGAAGAAGAGACAACGCGGCAGGGGGAGTCTGACGTCCGACCTCGACTCGAGAAGTCGCTCGCCAAAACGTGTGTGCAGTGTCTTCTGTTTCTTTCCAAGTTCTCCAGTCCGAAGAAACCGGACACTCTGACATGACTCGATACAGGGACCTGCCCGCCGACTCTTTCCTACTTTCAGCGGTCCTCCCTGTTCATCTTTCCTGTGACATTTCGGCATCTCTTTTTCTTGGTTTCCTCGCCTTCTCACCTGACTGAAGCCCCAGAAAAAGCCGAGGAGAGCCGCAGCGCGCTCTTCTTCCTTCAGCGTCCTCAGAAGAACGCTCTGGTACCGTTCCGTGAAGTGAGGCTCTAAACCGAACGCTGAAACAATGCGAATACCGTTCAGAGCCTCGCTCATCACGAAGGCAGCGGTGTCGCGGTCCTCCACCTTCTCCGCCTTCTTGTTCGCCCCCTCACCTGCAGAAAAAACTCCAAGGTTTCCAAAGCCTCGTGAGGCTCCCCATGAAGCTCTCCGCCTACGCTTGCGCAGATTGAGGCAGAGCAAACTACGCAAATGTGAGCCTACATGTACACACAGTTTCGTCGAGATTTGTACCTATATCTAAGAAGATTTGTACGGAAATGCGGGTGTGAAGCGGCAGTTTTCGAGGTGGCGTGCATACATCGACGCGACTCGGAGACCCAGCTTTGAGGAGACAGGAGAGAGAAAAGGAAACGGAGATAGAGCAGGTGGGGAGATCAGGTTTGCTCTGGGAGACGTGGACGGTCGCAGACGAAGAAGCAGACGCACGGAGCGAGCAGTGCAAAAAAGCGCGAGACAGAGCCGGCGCTGGGGAACTTCTGAGGAAGAATTCGAGAGAGAGAGGACCGTGGTGAAAAGCCAAGCTAAACGCGTGGACTTCCAGTTCTGCGGACTTTTCGGAGCCGAAATGTGAGACTGAGCGGCAGTGGCGGGGAGCAGAAGAACAAAACATGCGAGGACCACGGCGGCCAAGCGCGCGTCTCCAAAGAACGCGATGATGACACCTGAAATAAAGATAATGGAAAAAACAGAAGAAAAGCAACGGTCTCTCGCGCAGTCTTCAATCACGAGTTGACGCACACACGCATTAGGGGAGAACAACCTCTGCGATGTTGGATGCTTCTTTAGTGAGTGGACGTTTCCAGAAATCAAATCAAAGTAGCACGACACCGACAAGCAAAGAGATGTATACTTTCGTTCACGAGCACTGAAAGACGCGTCTAGACGCCTACGGATGCAGAGGGATCTGAAGCGACGAGTGAACAGTCAAAAAGCTTTCCGGCCTACCAGTGACAACAGCAGCCAATCCCTGGGTCATCGCGAGTGCGTTTCCAGCGCTTCCTGTCTTGACGAGAAGGACGTCGCTGGAGAGAACTCCCGTGAGATATCCTGAGGTCATTTTTTAGAAGAAGCGAACACTCTGGCGCGGCGTCTTCACTCTCGTGCTCACAGAAAGAATGAACTCACGAGACCCATGGCAGGACAAGTCTCAGACAGACACACAC"

This will find a single hit and display a link

You can use FILE > NEW FROM CLIPBOARD to bring the sequence into MacVector quickly

Again if you do not change the start coordinate of the sequence the IMPORT FEATURES dialogue will show an error

The dialogue will show that it has found 6 features. Note that the GFF3 file contains annotation from a much longer sequence than our initial query sequence. MacVector will ignore any annotations that lies outside the query sequence. It will show an warning message to indicate this.

Keeping a sequence updated

You will be able to “update” and add new annotation to your existing sequence. For example after a few months I could revisit these two genome browsers website and download an updated GFF3 file. Upon importing these features it will optionally replace any duplicate features and add new ones. So you can work with a sequence and also keep it updated as other researchers find more about this particular sequence.

Duplicate Features

Due to the lack of strict standards across the many different file formats it may be that a potential duplicate is not recognised as such because the wording or keyword is different. In the majority of cases some degree of manual curation of the annotated sequence will be required. In all cases MacVector will err on the side of caution and will never throw away any potentially interesting or important information contained within a feature. Only entries that are 100% the same (after being parsed during the import) will be considered as duplicates. MacVector will never class a feature as a duplicate if the START, STOP or FEATURE TYPE are different in any way. Even if they differ by just a single base.

Run the version check on this page to check which version of which part you are running and if you need to upgrade.

Remember if you are upgrading from a version earlier than K2 v 7 you’ll need a new MacVector license. Contact Support to request one.

Apple started transitioning their processor architecture from PowerPC (PPC) to Intel’s x86 in 2006. The Macbook Pro and the iMac being the first Intel Macs. When they did so they gave Xcode (the OSX development environment) the ability to create Universal Binary application. These contain native executables for both PPC and x86 CPUs. So giving these applications the ability to run natively on both types of Mac.

However, to support earlier applications that had only been available as PPC executables they included a PPC Emulator called Rosetta (named after the Rosetta Stone). This allowed PPC only applications to run on Intel x86 Macs.

OS X 10.5 Leopard was the last release of OS X that was distributed as a Universal Binary OS. With OS X 10.6 Snow Leopard being a x86 only release. That is it will not run on PPC Macs. Even so they kept Rosetta as an optional install (previously it was installed by default). However, OS X 10.7 Lion does not include Rosetta and you cannot install earlier versions. This means that PPC applications will no longer run on Lion. In reality this is only applications that are over six years old.

MacVector 9.5 (released in 2006) was the first Universal Binary version of MacVector. Previous versions are PPC only and will not run on Lion.

If you are running such an old version of MacVector then your options are limited to either upgrading MacVector or not upgrading to Lion. You can still upgrade such an old version of MacVector at a significant discount. Contact Sales to request a quote or visit the website. There’s many other reasons to upgrade as well. The interface has changed substantially since MacVector 9.5 being more powerful and easier to use. There’s also many new tools with enhancements to older ones. The latest release is MacVector 12.5 and this release has improvements to sequence assembly (support for NGS reference assemblies using Bowtie) and extra options for multiple sequence alignment with Muscle and T-Coffee added.

Technorati Tags: apple, lion, OSX, Universal Binary

Both T-Coffee and Muscle are progressive alignment algorithms as is ClustalW. Progressive alignments generally build a guide tree that represents the pairwise relationships between each possible pair of sequences in the alignment. A multiple sequence alignment is then built sequentially using the tree as a construction guide. T-Coffee builds a library of all pairwise alignments but also aligns each sequence in the pair with a third sequence in the sequence set before building the MSA. Muscle does not do a pairwise alignment but instead uses an approximate method of comparing the number of short subsequences (k-mers, k-tuples or words) that each pair of sequences share. You can immediately see how this is much faster for alignments containing many sequences where the number of pairwise alignments needed to construct the tree is high.

T-Coffee is regarded as being slightly slower than ClustalW but will produce more accurate alignments for distantly related amino acid sequences. Here’s the original publication. Incidentally T-Coffee stands for Tree based Consistency Objective Function For AlignmEnt Evaluation.

Muscle is generally regarded as faster than Clustalw and T-Coffee at the penalty of being slightly less accurate.

All three algorithms are integrated into the MSA editor. This means you can try all three algorithms on the same alignment to see the results.

Here’s a few benchmark for protein sequence alignments (the files are in the Sample Files folder in the MacVector application folder:

- Clustalw: 1min26secs

- Muscle: 12 seconds

- T-Coffee: 17mins22secs

Because of the initial step of constructing the pairwise alignments tree, progressive alignment algorithms have difficulties with alignments where all sequences do not share a common region. For example take an alignment of three sequences where you have 5kb sequence and two regions of this sequence of around 1KB long. One subsequence aligns from 1,000 to 2,000 of the 10kb sequence and the other aligns at 4,000 to 5,000. Most progressive alignments will try to create initial pairwise alignments of all combinations of sequences and that skews the alignment so that it prefers to align the sequences so that there is at least one segment of overlap between all of the input sequence. Due to not creating pairwise alignments the “Muscle” algorithm is unique amongst these three algorithms as it will align this type of data as long as the “Diagonals” optimization parameter is set to “On”.

Technorati Tags: Sequence Alignments, MacVector

So after you’ve chatted about the holidays with your colleagues have a slow start to 2012 and take the opportunity to keep your copy of MacVector up to date. MacVector 12.5 was released at the end of last year. So if you’ve not yet done so then do so here! It’s also a good idea to run a few system maintenance tasks to make sure your Mac is running great.

Happy New Year!

We have a lot to look forward to in 2012. MacVector 12.6 will be released in the first quarter of 2012 with some great new primer design tools. We’ve also got another release planned for the backend of the year.

But for now, we thank you and we wish you a prosperous, happy and enjoyable New Year from the whole MacVector team.

But before we all break for the end of the year remember that we’ve just released MacVector 12.5. You’ve probably seen the release via Software Update inside MacVector but if not you can download it from our website if your license is valid for this release or download a 21 day trial.

Technorati Tags:MacVector

This is a major release with a lot of new features. Many parts of MacVector have been substantially rewritten both to keep up with OS X as well as many dialogs that have been redesigned.

The main new features are for sequence alignments. We’ve added Bowtie to Assembler for further support of NGS data assemblies. Along with variant detection using Samtools to match the existing SNP report tab.

We’ve also added additional alignment algorithms. Both Muscle and T-coffee have been added to the Multiple Sequence Alignment analysis interface, complementing the existing ClustalW algorithm. We’ve wanted both of these for a while now and judging from the results of last year’s survey so have many users.

There’s also enhanced support for segmented features as well as being able to display translated CDS features directly in the sequence editor as well as the Map view. There’s SNP reporting for the Align to Reference editor as well as greatly improved performance for its algorithm and also for the Auto Annotation tool.

MacVector 12.5 is fully Mac OS X 10.7 (Lion) compatible as well as being supported on OS X 10.5 (Leopard) and OS X 10.6 (Snow Leopard). However, MacVector 12.5 is not supported on OS X 10.4 (Tiger). If you are still using Tiger (from our last survey much less than 1% of our users still use Tiger) MacVector 12.0.6 will still remain supported.

This is just a short overview, if you have a valid license download the updater or if you do not download a 21 day trial.

Technorati Tags:MacVector

With each release of MacVector as well as adding new features we continually redesign and rewrite many existing parts of MacVector. This is done for a number of reasons. For example:

As a recent example you can now run a primer analysis and literally drag and drop a primer pair or their product to the sequence window to annotate those features to the sequence.

Here’s a few of those general improvements that have been included in MacVector 12.5.

Interface improvements

Sequence Window Enhancements

Miscellaneous Enhancements

..and remember if you want to add Assembler before MacVector 12.5 is released you can do so at half price. You can also purchase an upgrade or a new license before the release of MacVector 12.5 and get a 10% discount. Both these offers include a free upgrade to MacVector 12.5 when it is released. Please request a quote now. Don’t forget to quote the promotional code of “Assembler50%” or “MV12510″ for the discounts.

Technorati Tags: bowtie, MacVector

With Assembler 12.5 our developers have come up with an affordable and straightforward solution for assembling and visualizing your NGS data. Generating sequencing data is cheaper than it has ever been, however, with this increase in data has come a problem with analysis. Assembler will now create reference assemblies with just a few mouse clicks using Bowtie. Instead of sending your millions of reads away to be assembled or delving into complicated software tools you’ll be able to align millions of NGS reads to multi megabase reference sequences in literally minutes. Bowtie is a fast algorithm, and although it’s an ungapped assembler, what it loses in accuracy it makes up for in speed. You do not need a 32GB 8core Mac Pro to assemble your data. In addition to the existing phrap/phred tools this makes Assembler a simple, cost-effective solution to analyzing your Next Generation Sequencing reads.

Creating a Reference Assembly

Your reference sequence can be in any “openable” format. However, your reads need to be in FASTQ format.

Hit Reporting

In the dialogue you’ll see an important setting called Hit Reporting. Bowtie uses a concept of strata to score alignments. A stratum is defined by all reads that contain the same number of mismatches in the seed (the seed is the first “n” bases of a read which is given higher priority in scoring than the entire read). You can either show ALL ALIGNMENTS, REPORT BEST ALIGNMENT ONLY (show the best alignment in the stratum with the least amount of mismatches) or REPORT ALL BEST ALIGNMENTS (which shows the best alignment in all strata). Which you choose depends on a few factors. For example how many references you have, how many repeated regions you expect, whether you are using a reference sequence from the same organism or a related one, and many others. Generally start with show all alignments, which is the quickest, and work from there.

Analysis

..and that’s how easy it is. Of course generating results is always easier than analysing them and to help analyse your reference contig Assembler has a few useful tools. We’ll talk about variant detection in a later blog post, but the coverage map is one of the first tools that you will see upon completing an assembly.

Using the Coverage Map

It is extremely useful to be know the depth of reads that are aligned on your reference. Areas of low coverage indicate that you need further sequencing and peaks of high coverage can be indicative of repeats. The Map view of a reference contig will show details of the depth of reads in a coverage map with four statistics. A single plot line shows a running average of the number of reads at that point. However, an average plot is not very sensitive when viewed at a high level and so two shaded areas indicate the maximum value and the minimum value of the averaged reads at that point. As the coverage map is viewed at a lower level these three values will become increasingly closer to the extent that when viewed at, or close to, residue level these three plots will become identical. Areas of zero coverage are shown in light grey. Note that these areas are always displayed even when they are disproportionate to the level of magnification.

Multiple reference sequences

You can add multiple reference sequences and depending on the settings reads will be aligned against the best match or against multiple ones. This is great for such tasks as identifying a sequenced isolate amongst a series of closely related strains of virus or bacteria. Having multiple reference sequences helps determine which is most closely related (or identical) to the isolate.

Paired end reads

Paired end reads are very useful for improving the accuracy of alignments and also for indel detection. Paired end reads are created by sequencing both ends of the same DNA molecule, with known fragment size. Since the two reads are now separated by a known distance assembly and orientation of the two reads is less complicated. For Assembler if your reads are paired end all you need to do is ensure that the same filenames but appended with version numbers and Paired End assembly is enabled.

e.g.

READS_1.fastq
READS_2.fastq

You’ll also need to input the fragment size.

In the next Assembler post we’ll talk about variant detection.

..and remember if you purchase an upgrade or a new license before the release of MacVector 12.5 you can get Assembler with a 50% discount and a free upgrade to MacVector 12.5 when it is released. This offer ends on 1st December. Please request a quote now. Don’t forget to quote the promotional code of “Assembler50%”

Technorati Tags: bowtie, MacVector

However, the most important new feature in MacVector 12.5 is the addition of Bowtie to MacVector’s sequence assembly module, Assembler to create reference assemblies with your NGS data. Additionally, the Contig Editor has been enhanced with some useful features to visualise your genome sized alignments. With these improvements, our developers have come up with an affordable and straightforward solution to assembling your NGS data. Generating sequencing data is cheaper than it has ever been, however, analyzing it is not. Instead of sending your millions of reads away to be assembled at great expense, or spending time on the computer trying to assemble your reads, Assembler will now create reference assemblies with just a few mouse clicks.

To mark the release of MacVector 12.5 and this great new functionality in Assembler we’re offering users an opportunity to add Assembler to your MacVector license for half price. Not only that but if you have an older license of MacVector and want to upgrade your license then you’ll still be able to add Assembler at half price. If you do not have MacVector and want this great new functionality then you’ll still be able to add Assembler to a new license for half price.

If you take advantage of this offer, you will not only get a great price on Assembler, but you will get MacVector 12.5 and a free upgrade to MacVector 12.6 a few months after. MacVector 12.6 will include even more NGS support as well as a Quick-test function to easily analyze your existing primer sequences, and more. Plus, your license will include a year’s maintenance which will not start until MacVector 12.5 is officially released on the 1st December, 2011.

Remember that this half-price offer ends on 1st December, and the price of Assembler is going up next year. Please request a quote now. Don’t forget to quote the promotional code of “Assembler50%”

If you want to see the new functionality then we have a prerelease preview available for download. This is not yet a release candidate but is part of our external beta program

..and remember both MacVector 12.0.6 and MacVector 12.5 are fully supported on OS X Lion.

MacVector 12.5 will be released on the 1st of December, 2011

Technorati Tags: MacVector