Difference between revisions of "MCarts Documentation"
From Zhang Laboratory
(→Prediction) |
(→Prediction) |
||
| Line 180: | Line 180: | ||
| − | This will produce the results in XXX, in bed format. The 5th column is the motif cluster score. This file can be converted into bedGraph, which in combination with the individual motif sites, will give the best visualization. | + | This will produce the results in XXX, in bed format. The 5th column is the motif cluster score, which can be used to rank the clusters. High scoring clusters are in general more reliable than low scoring clusters. |
| + | |||
| + | This file can be converted into bedGraph, which in combination with the individual motif sites, will give the best visualization. | ||
=Additional options= | =Additional options= | ||
Revision as of 14:41, 20 September 2012
Prediction of clustered RNA-binding protein motif sites in the mammalian genome
Chaolin Zhang,1,* Kuang-Yung Lee2,3, Maurice S. Swanson2, Robert B. Darnell1,*
1 Laboratory of Molecular Neuro-Oncology, Howard Hughes Medical Institute, The Rockefeller University, 1230 York Avenue, New York, NY 10021, USA 2 Department of Molecular Genetics and Microbiology and the Center for NeuroGenetics, University of Florida, College of Medicine, Gainesville, FL 32610, USA 3 Department of Neurology, Chang Gung Memorial Hospital, Keelung, Taiwan
* Corresponding authors
Contents
Introduction
mCarts is a hidden Markov model (HMM) based methods to predict clusters RNA motif sites.
Many RBPs recognize very short and degenerate sequences, with targeting specificity achieved by mechanisms such as synergistic binding to multiple clustered sites and modulation of site accessibility through different RNA-secondary structures. mCarts integrates the number and spacing of individual motif sites, their accessibility and conservation, which substantially improves signal to noise ratio. This algorithm learns and quantifies rules of these features, taking advantage of a large number of in vivo RBP binding sites obtained from high throughput sequencing of RNAs isolated by cross-linking and immunoprecipitation (HITS-CLIP). We applied this algorithm to study two representative RBPs, Nova and Mbnl. Despite the very low information content in individual motif elements, our algorithm made very specific predictions for successful experimental validation.
Download
Source code:
- czplib (perl): a perl library with various functions for genomic/bioinformatic analysis. (download from SourceForge.net)
- mCarts (perl): the core algorithm. (download from SourceForge.net, need permission now)
- PatternMatch (c/c++): a handy tool to search individual motif sites based on consensus. It supports degeneracy and mismatches. (download from SourceForge.net)
- RegExpMatch (c/c++): a handy tool to search individual motif sites based on regular expression (download from SourceForge.net)
Library data:
- mm9 (15 Gb compressed /109 Gb uncompressed): download
- hg18 (36 Gb compressed /212 Gb uncompressed): download
Nova training data:
1.8 Mb compressed: download
Installation
Prerequisite
This software is implemented with perl and c/c++. It also relies on several standard linux/unix tools such as grep, cat, sort, etc. We have tested the software on RedHat Linux, although it is expected to work on most unix-like systems, including Mac OS X.
Steps to install the software
- Download the perl library files czplib, if not already.
Decompress it and move it to a place you like
tar zxvf czplib.v1.0.0.tgz mv czplib /usr/local/lib
Add the library path to the environment variable, so perl can find it.
PERL5LIB=/usr/local/lib/czplib
- Download mCart codes, if not already.
Decompress it and move it to a place you like
tar zxvf mCarts.v1.0.0.tgz cd mCarts chmod 755 *.pl mv mCarts /usr/local/mCarts
Add the dir to your $PATH environment variable.
- Download and compile PatternMatch and RegExpMatch
tar zxvf PatternMatch.v1.0.0.tgz cd PatternMatch make chmod 755 PatternMatch mv PatternMatch /usr/local/bin tar zxvf RegExpMatch.v1.0.0.tgz cd RegExpMatch make chmod 755 PatternMatch mv RegExpMatch /usr/local/bin
Make sure /usr/local/bin is already in your $PATH
- Download and decompress library files
tar zxvf mCart_lib_data_mm9.tgz mv mCart_lib_data_mm9 /home/czhang/data/mCart_lib_data_mm9
Get started
mCarts requires genomic sequences of genic regions, multiple alignments, exon/intron/UTR annotations, RNA accessibility information, etc, which are all provided in the library files (that's why it is huge!!).
In addition, mCarts takes two sets of genomic regions, provided in two BED files, to obtain motif sites in the positive and negative training datasets of the HMM. The positive training regions are typically (several thousand) regions of robust CLIP tag clusters. The negative training regions are typically genic regions without any CLIP tags. Only motif sites in genic regions (as defined by library files) are actually used.
A real example to predict Nova binding YCAY clusters.
To get training data, download.
There are two files in the compressed package: mm9.Nova.train.pos.bed specifies 6,231 non-repetitive, genic Nova CLIP tag clusters with peak height (PH)≥15, and located in exons or 1 kb flanking intronic sequences on each side (exon+ext1k sequences). mm9.Nova.train.neg.bed specifies 110,998 exon+ext1k sequences, in which no CLIP tags were present.
It is recommended to divide the process to two steps, although they can also be combined into one single step:
Model training
mCarts -v -ref mm9 -w YCAY -f ./Nova.train.pos.bed -b ./Nova.train.neg.bed -lib /home/zhangc/data/mm9_mammal_input_data --train-only ./mm9_Nova_out
This command specifies the verbose mode (-v), reference genome (-ref mm9), the consensus motif to search (-w YCAY), foreground or positive training regions (-f ./Nova.train.pos.bed), background or negative training regions (-b ./Nova.train.neg.bed), directory with library files (-lib /home/zhangc/data/mm9_mammal_input_data), model training only (--train-only), and the output dir (./mm9_Nova_out)
see a complete list of options.
This command will do the following things:
- Search for individual motif sites in the genic regions (with 10 kb extension on each side) in the reference genome (mm9) and many additional genomes (e.g., 19 other mammalian genomes aligned to mm9) and evaluate their conservation using branch length scores (BLS).
- Retrieve the RNA accessibility information as measured by probability of unpairedness (PU) or single strandedness. These scores for all tetramers in genic regions were pre-calculated, because calculating PU scores is quite slow.
- intersect motif sites with positive and negative training regions to get training motif sites for the HMM
- estimate parameters of the HMM.
The following out put files are of particular interest:
- model.txt:
This file saves parameters of the HMM, including emission probabilities and data to calculate transition probabilities. The emission probability is represented by histograms (nonparametric), and can show how each feature contrast RBP bound motif sites vs. background motif sites. This information is specified in the following lines:
distance_positive 3.59E-20 3.59E-20 3.59E-20 0.078482739 ... distance_negative 2.90E-22 2.90E-22 2.90E-22 0.044107727 ... # 0 0.05 0.1 0.15 ... conservation_positive_0 0.179478553 0.182674516 1.68E-19 0.023044575 ... conservation_positive_1 0.02999663 0.047691271 1.69E-19 0.005729693 ... conservation_positive_2 0.101576182 0.126094571 8.76E-19 0.014886165 ... conservation_positive_3 0.12325902 0.163378809 4.75E-20 0.019869753 ... conservation_negative_0 0.341826256 0.282508127 3.69E-22 0.033694368 ... conservation_negative_1 0.11093915 0.126552235 2.04E-21 0.01781323 ... conservation_negative_2 0.237788887 0.22985303 6.83E-21 0.022994864 ... conservation_negative_3 0.291355883 0.259434326 4.82E-21 0.029970126 ... accessibility_positive 0.014387222 0.057225909 0.073668448 0.082271419 ... accessibility_negative 0.059879224 0.132628229 0.118651224 0.106571071 ...
The probability of distance are for spacing from 0,1,2, ...
Conservation scores and accessibility scores are in the range between 0 and 1. For conservation scores, the suffix represents different regions (0-intron, 1-CDS, 2-5'UTR, 3-3'UTR).
This information can be used to produce a figure as shown below:
Note that the distance distribution for the positive training sites is censored at 30 nt (--max-dist 30, default). For Nova, all features contribute to the discrimination of Nova bound YCAYs and background sites.
- BLS/mm9.genic.ext10k.motif.bls.chrom.bed
This is the bed file of individual motif sites, which could be quite big if the motif is short and degenerate (like Nova). You can load this file to the genome browser for visualization. The 5th column if each row is the BLS conservation score of each site, which is in the range of 0 and 1, and should be re-scaled to 0-1000 for the best visualization.
awk '{print $1"\t"$2"\t"$3"\t"$4"\t"$5*1000"\t"$6}' BLS/mm9.genic.ext10k.motif.bls.chrom.bed > mm9.genic.ext10k.motif.bls.chrom.2.bed
Prediction
After the model is generated, and verified (or even modified if you like by editing model.txt) to make sense, one can move forward for prediction using the command line below.
mCarts -v --exist-model ./mm9_Nova_out
This will produce the results in XXX, in bed format. The 5th column is the motif cluster score, which can be used to rank the clusters. High scoring clusters are in general more reliable than low scoring clusters.
This file can be converted into bedGraph, which in combination with the individual motif sites, will give the best visualization.
Additional options
Running jobs in parallel
Don't be surprised. Support for running jobs in parallel using a queue system (well, at least for SGE) is already included. The program will try to find if SGE is available. If yes, jobs will be dispatched to the default queue, and the results will be collected and combined when all jobs are done. Otherwise, the job will run locally.