Tagged with #realignment
0 documentation articles | 0 announcements | 11 forum discussions

No articles to display.

No articles to display.

Created 2015-12-21 15:10:30 | Updated | Tags: realignertargetcreator dbsnp realignment gatk random-chromosomes

Comments (7)

I'm re-aligning some bam files using RealignerTargetCreator. I downloaded the hg38 dbSNP file and hg38 reference genome. however, the dbSNP file contains the canonical chromosomes [1,2...X,Y,MT] while the reference (and my bam file) contain also the random chromosomes.

here's where I get the error: MESSAGE: Input files and reference have incompatible contigs: No overlapping contigs found. 142_hg38.vcf contigs = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, X, Y, MT] reference contigs = [chr1, chr1_GL383518v1_alt, chr1_GL383519v1_alt, chr1_GL383520v2_alt ... [etc etc etc.....] ... chrX_KI270881v1_alt, chrX_KI270913v1_alt, chrY, chrY_KI270740v1_random, chrM]

so...I would like to keep the random chromosomes in the reference/bam file, so how can I have this run without tossing an error? or is this not possible and I need to keep ONLY the chrs in the dbSNP file?

thanks Seb

Created 2014-05-16 21:22:48 | Updated | Tags: baserecalibrator vcf realignment

Comments (5)


I am following the gatk best practices guide, and using the 2 stages BaseRecalibrator process. The first recalibration is going well, the second one, produces an error like described below, I find it strange since it is not mentioned in the documentation (neither the tutorials that BaseRecalibrator require a vcf input

Here is my command

java -Xmx2g -jar GenomeAnalysisTK.jar \
-T BaseRecalibrator \
-R GRCh37-lite.fa \
-I SA495-Tumor.sorted.realigned.bam \
-BQSR SA495-Tumor.sorted.realigned.grp \
-o SA495-Tumor.sorted.post_recal.grp2

here is the error message

INFO  14:03:38,425 GATKRunReport - Uploaded run statistics report to AWS S3
 ERROR A USER ERROR has occurred (version 3.1-1-g07a4bf8):
 ERROR This means that one or more arguments or inputs in your command are incorrect.
 ERROR The error message below tells you what is the problem.
 ERROR If the problem is an invalid argument, please check the online documentation guide
 ERROR (or rerun your command with --help) to view allowable command-line arguments for this tool.
 ERROR Visit our website and forum for extensive documentation and answers to
 ERROR commonly asked questions http://www.broadinstitute.org/gatk
 ERROR Please do NOT post this error to the GATK forum unless you have really tried to fix it yourself.
 ERROR MESSAGE: Invalid command line: This calculation is critically dependent on being able to skip over known variant sites. Please provide a VCF file containing known sites of genetic variation.

Any idea ?



Created 2013-12-17 19:29:51 | Updated | Tags: indelrealigner realignertargetcreator realignment

Comments (4)


I'm indel realigning with version 2.4-9 using generic commands such as:

java -Xmx4g -jar /path/to/GenomeAnalysisTK.jar \ -T RealignerTargetCreator \ -R /path/to/reference.fasta \ -I /path/to/input.bam \ -o /path/to/realigner.intervals

java -Xmx4g -jar /path/to/GenomeAnalysisTK.jar \ -T IndelRealigner \ -R /path/to/reference.fasta \ -I /path/to/sample-level.bam \ -targetIntervals /path/to/realigner.intervals.from.rtc \ -o /path/to/realigned.bam \ -model USE_SW \ -LOD 0.4

In most cases this is working fine, but in a few cases it is introducing artefacts that subsequently cause the bam file to fail Picard's ValidateSamFile, and in a couple of cases it introduces errors that can't be fixed by CleanSam and/or FixMateInformation.

Here's an example of an error that can be fixed by CleanSam:

Before indel realignment:

HS24_08564:7:2311:4630:19372#87 69 AAKM01002546 471 0 * = 471 0 HS24_08564:7:2311:4630:19372#87 137 AAKM01002546 471 25 100M = 471 0

After indel realignment:

HS24_08564:7:2311:4630:19372#87 69 AAKM01002546 471 0 * = 471 0 HS24_08564:7:2311:4630:19372#87 137 AAKM01002546 471 35 91M1D9M = 471 0

Here's an example of an error that can't be fixed:

Before indel realignment:

HS24_10061:6:1312:10172:98346#54 69 AAKM01002280 649 0 * = 649 0 HS24_10061:6:1312:10172:98346#54 137 AAKM01002280 649 37 100M = 649 0

After indel realignment:

HS24_10061:6:1312:10172:98346#54 69 AAKM01002280 649 0 * = 649 0 HS24_10061:6:1312:10172:98346#54 137 AAKM01002280 649 47 91M2D9M0S = 649 0

Is this a known bug? Any chance of a fix?



Created 2013-08-23 14:16:01 | Updated | Tags: realignertargetcreator realignment incompatible-contigs

Comments (1)


I am currently trying to run the RealignerTargetCreator on some bam files which were aligned to hg19 howver am getting this error `ERROR MESSAGE: Input files known and reference have incompatible contigs: Found contigs with the same name but different lengths:

ERROR contig known = chrM / 16571
ERROR contig reference = chrM / 16569.`

After some initial investigation I found that the supplied hg19 reference genome which was being used for mapping was using rCRS mtDNA. other then realigning to a different build of hg19 is there any way to easily fix this problem through GATK?

Created 2013-07-09 02:16:17 | Updated | Tags: indelrealigner realignertargetcreator realignment

Comments (5)

I am doing exome sequencing in 700 individuals from a species with a large genome and I would like to use GATK to realign around indels. I am using a reduced reference, which is still about 3Gb. I tested out the target creator, but it is taking 5 days for 12 individuals when each is done individually and this time frame is not feasible for all 700 individuals. I tried to run more in parallel (~30 individuals), but there are RAM limitations on our 250G server. I am currently testing out the program by running all 12 test samples as input for the same run and the time estimate is very long (on the order of several hundred weeks). Based on a preliminary run I have also included a vcf file with likely indels to try and speed the process. Can you suggest another way in which I can make the time frame for all 700 individuals more reasonable? Otherwise we will not be able to use this tool.

Created 2013-07-04 12:58:28 | Updated | Tags: realignment

Comments (17)

Hi, I'm currently working with bwa, samtools and GATK to make SNP calling on Medicago truncatula. I'm using my own reference sequence, with the 8 chromosoms in the same fasta file.

C1_lenght=155648 AAAGATAGAGA.. C2_lenght=125018 ATGGATC... etc.. I have done alignments without problem, but for GATK : I do rmdup --> CreateSequenceDictionary.jar (picard) --> samtools sort --> Read Group (picard) --> samtools index and then : Pre alignment with :

java -jar -Xmx4g /usr/local/bioinfo/src/GATK/GenomeAnalysisTK-2.4-9-g532efad/GenomeAnalysisTK.jar -nt 8 -T RealignerTargetCreator -R REF.fa -o RTC.intervals -I INPUT_muq30_RMDUP_RG.bam

Here there is no problem, but when I want to make the realignement :

java -jar -Xmx4g /usr/local/bioinfo/src/GATK/GenomeAnalysisTK-2.4-9-g532efad/GenomeAnalysisTK.jar -T IndelRealigner -R REF.fa -I INPUT_muq30_RMDUP_RG.bam -targetIntervals RTC.intervals -o INPUT_muq30_RMDUP_RG_REAL.bam

And I got this error message : ERROR MESSAGE: Bad input:We encountered a non-standard non-IUPAC base in the provided reference: '13'

I didn't find any explanation in google for this error. Could you please help me ?!


Created 2013-03-13 12:13:22 | Updated 2013-03-13 12:40:59 | Tags: realignertargetcreator baserecalibrator realignment

Comments (13)


I am currently working with a project where we have sequenced a library of approximately 70 bps insert sizes using 2x100 paired-end seq. While this can seem unnecessary, it can improve base qualities a lot.

I have used SeqPrep (https://github.com/jstjohn/SeqPrep) which strips adaptors and merges reads that overlap, in our case the entire read most of the times. This also boosts the base qualities, if a base was sequenced twice, the quality improves quite a bit. This way, base qualities can stretch up to 70 and over (probability of error 0.0001 x 0.0001 if both reads had Q40 at that base, it merged qual = 80). No funny business there. :)

However, this does not seem to play nicely with GATK. The realignment crashes (see below) saying the the base quals must be erroneous. In my case, but they are correct. Can I force GATK to work with these BQs? (--validation_strictness LENIENT didn't help as you can see below :)

cheers Daniel Klevebring

   INFO  13:04:07,408 HelpFormatter - -------------------------------------------------------------------------------- 
   INFO  13:04:07,411 HelpFormatter - The Genome Analysis Toolkit (GATK) v2.4-7-g5e89f01, Compiled 2013/03/06 01:01:28 
   INFO  13:04:07,411 HelpFormatter - Copyright (c) 2010 The Broad Institute 
   INFO  13:04:07,411 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk 
   INFO  13:04:07,416 HelpFormatter - Program Args: -T RealignerTargetCreator -I /scratch/3041404/P394_102.prmdup.bam -R /bubo/proj/b2010040/private/GoldenPath/hg19/GATK_resource_bundle/human_g1k_v37_clean.fasta -o /scratch/3041404/P394_102.realn.intervals --intervals /bubo/proj/b2010040/private/GoldenPath/NG_design/1000G_REF_picard_custom_design_target_regions_HG19.bed.interval_list --validation_strictness LENIENT 
   INFO  13:04:07,416 HelpFormatter - Date/Time: 2013/03/13 13:04:07 
   INFO  13:04:07,416 HelpFormatter - -------------------------------------------------------------------------------- 
   INFO  13:04:07,416 HelpFormatter - -------------------------------------------------------------------------------- 
   INFO  13:04:08,461 GenomeAnalysisEngine - Strictness is LENIENT 
   INFO  13:04:08,632 GenomeAnalysisEngine - Downsampling Settings: Method: BY_SAMPLE, Target Coverage: 1000 
   INFO  13:04:08,640 SAMDataSource$SAMReaders - Initializing SAMRecords in serial 
   INFO  13:04:08,655 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.01 
   INFO  13:04:09,782 IntervalUtils - Processing 39772003 bp from intervals 
   INFO  13:04:10,001 GenomeAnalysisEngine - Creating shard strategy for 1 BAM files 
   INFO  13:04:10,262 GenomeAnalysisEngine - Done creating shard strategy 
   INFO  13:04:10,263 ProgressMeter -        Location processed.sites  runtime per.1M.sites completed total.runtime remaining 
   INFO  13:04:18,482 GATKRunReport - Uploaded run statistics report to AWS S3 
   ##### ERROR ------------------------------------------------------------------------------------------
   ##### ERROR A USER ERROR has occurred (version 2.4-7-g5e89f01): 
   ##### ERROR The invalid arguments or inputs must be corrected before the GATK can proceed
   ##### ERROR Please do not post this error to the GATK forum
   ##### ERROR
   ##### ERROR See the documentation (rerun with -h) for this tool to view allowable command-line arguments.
   ##### ERROR Visit our website and forum for extensive documentation and answers to 
   ##### ERROR commonly asked questions http://www.broadinstitute.org/gatk
   ##### ERROR
   ##### ERROR MESSAGE: SAM/BAM file SAMFileReader{/scratch/3041404/P394_102.prmdup.bam} appears to be using the wrong encoding for quality scores: we encountered an extremely high quality score of 70; please see the GATK --help documentation for options related to this error
   ##### ERROR ------------------------------------------------------------------------------------------

Created 2012-10-22 08:23:25 | Updated 2013-01-07 20:12:22 | Tags: realignertargetcreator bwa realignment

Comments (1)


before I only used BWA and as you described in the best pratice I performed the realign step. Now I want to integrate in my pipeline Stampy associated with BWA.

Do you think, I should make the realign step ?

Thanks !

Created 2012-10-10 15:49:51 | Updated 2012-10-18 01:17:18 | Tags: indels fixmate realignment

Comments (5)

I recently used GATK, the latest version to carry out Indel realignment for my bam files. I have read at few places that one needs to fix the mate position information as the reads mapping position may change during realignment. My question is that does this step is taken care of y GATK automatically or one needs to run Picard fixmates before going for subsequent analysis.


Created 2012-09-21 10:51:56 | Updated 2013-01-07 20:11:25 | Tags: fixmate realignment

Comments (9)

Dear Community and GATK's team,

I have one question about the cleaning step before SNP calling, mainly about local realignment around indels.

I read on some website describing their workflow that alignments may change during the realignment process, it would prefer to fix the mate information and Picard offers this utility to do that for us. is it true? Or are there only any insert sizes that can change? If there are some change of insert sizes, is there a tool that checks that these changes are ok?

What do you use Picard's tool, FixMateInformation.jar, to fix the mate information when using paired-end data ?

Up to now, I have not used in my pipeline. Maybe this is a mistake. If we have to add this step, should we add this step after the realignment step or recalibration step?

Thank you for your help,


Created 2012-08-29 10:02:02 | Updated 2013-01-07 20:42:47 | Tags: best-practices readgroup realignment baserecalibration

Comments (3)


I am new at using GATK (v 2.1-3). I do exome sequencing by sample using the following steps: Alignment with BWA (0.6.2) GATK :Local realignment around INDELs PICARD (1.67): FixMateInformation GATK: Recalibration (BaseRecalibrator + PrintReads -BQSR) Samtools for calling variants

Samtools seems to run properly but no file (.vcf and .bcf) are created and no error message is prompted :

cd Sample_09

  • samtools mpileup -BE -ug -q 20 -Q 20 -D -f human_g1k_v37.fasta realigned_fixed_recal.bam -C50
  • bcftools view -bvcg - [mpileup] 1 samples in 1 input files Set max per-file depth to 8000 [bcfview] 100000 sites processed. [afs] 0:89274.054 1:6411.053 2:4314.893 [bcfview] 200000 sites processed. [afs] 0:89100.642 1:6125.883 2:4773.474 [bcfview] 300000 sites processed. [afs] 0:87374.996 1:7439.238 2:5185.766 [bcfview] 400000 sites processed. [afs] 0:87890.186 1:7087.628 2:5022.185 [bcfview] 500000 sites processed. [afs] 0:85322.061 1:8454.843 2:6223.096 [bcfview] 600000 sites processed. [afs] 0:85864.368 1:8410.777 2:5724.854 [bcfview] 700000 sites processed. [afs] 0:88813.814 1:6828.001 2:4358.185 [bcfview] 800000 sites processed. [afs] 0:89070.318 1:6302.924 2:4626.758 [bcfview] 900000 sites processed. [afs] 0:88364.380 1:6796.962 2:4838.658 [bcfview] 1000000 sites processed. [afs] 0:86892.531 1:7268.088 2:5839.381 [bcfview] 1100000 sites processed. [afs] 0:87184.845 1:7153.073 2:5662.081 [bcfview] 1200000 sites processed. [afs] 0:86762.756 1:7241.236 2:5996.008 [bcfview] 1300000 sites processed. [afs] 0:89346.143 1:6159.989 2:4493.868 [bcfview] 1400000 sites processed. [afs] 0:88658.355 1:7160.555 2:4181.089 [bcfview] 1500000 sites processed. [afs] 0:85985.305 1:8308.039 2:5706.656 [bcfview] 1600000 sites processed. [afs] 0:87346.636 1:7708.883 2:4944.480 [afs] 0:63097.202 1:3950.127 2:3572.670
  • bcftools view .bcf
  • cd ..

I have seen that some groups use after realignment Picard:AddOrReplaceReadGroups and I wonder if I should use before calling the variant with samtools.

Thanks in advance for any advice you can give me.