gSearch - Performance

Huge Variant and Reference Files Test

Input Files

	gSearch		ANNOVAR		BEDTools
	exact search	range search	exact search	range search	range search
# of lines (data only)	52,814,665	52,557,645	52,814,665	52,557,645	52,557,645
File size	4.5GB	4.5GB	3.7GB	3.7GB	4.5GB

exact search : kaviar.111121.hg19.gvf
range search : kaviar.111121.hg18.gvf

Reference Files

	gSearch		ANNOVAR		BEDTools
	exact search	range search	exact search	range search	range search
# of lines (data only)	39,706,715	3,807,839	39,706,715	3,807,839	3,807,839
File size	6.6GB	155MB	1.2GB	182MB	184MB

exact search : ALL.wgs.phase1_release_v3.20101123.snps_indels_sv.sites
range search : Conserved transcription factor binding sites

Execution Time Summary

	gSearch		ANNOVAR		BEDTools
	exact search	range search	exact search	range search	range search
Execution Time (sec)	347	148	4,001	1,276	13,800

exact search: ./gsearch -i kaviar.111121.hg19.gvf -o kaviar.111121.hg19.af_annotated.gvf -r ALL.wgs.phase1_release_v3.20101123.snps_indels_sv.sites.vcf -rf VCF -tg allele_freq_1000g -m e -s n -t s -a y -l n -v n -nt 4 -b
range search: ./gsearch -i kaviar.111121.hg18.gvf -o kaviar.111121.hg18.tf_annotated.gvf -r hg18.ref.tfbs.txt -rf GSF -tg tfbs -m o -s n -t s -a y -l y -v y -nt 4 -b

PC Specification

Product	iMac MB953KH/A
CPU	Intel Core i5 2.66GHz
# of cores	4
Memory	4GB (2GB * 2, 1067 MHz DDR3)

Typical Whole Genome Sequencing Variant File Test

Input Files

A dataset of 17 whole-genome sequences (CEPH/Utah Pedigree 1463 data). See Download page for more information.

Reference Files

	gSearch		ANNOVAR
	Exact match (1000g2010_CEU)	Range search (sec) (tfbs)	Exact match (1000g2010_CEU)	Range search (sec) (tfbs)
# of lines (data only)	7,725,663	3,807,839	7,725,663	3,807,839
File size	265MB	155MB	160MB	182MB

PC Specification

Product	iMac MB953KH/A
CPU	Intel Core i5 2.66GHz
# of core	4
Memory	4GB (2GB * 2, 1067 MHz DDR3)

Execution Time Summary

Genome	Operation	gSearch (single-threaded)		gSearch (multi-threaded)		ANNOVAR
Genome	Operation	Exact search (sec)	Range search (sec)	Exact search (sec)	Range search (sec)	Exact search (sec)	Range search (sec)
Average (3,845,133 lines, 381MB)	Read data	11.58	7.37	11.59	7.39	50.57	51.17
	Sort	0.81	0.50	0.41	0.26	81.45	91.18
	Search	1.72	0.75	0.64	0.25
	Write results	5.02	4.16	4.85	3.97
	Sum	19.14	12.79	17.49	11.87	132.02	142.35

Execution Time Details

Genome	Operation	gSearch (single-threaded)		gSearch (multi-threaded)		ANNOVAR
Genome	Operation	Exact search (sec)	Range search (sec)	Exact search (sec)	Range search (sec)	Exact search (sec)	Range search (sec)
GS12877 (3,830,403 lines, 380MB)	Read data	11.46	7.30	11.43	7.24	50.55	51.10
	Sort	0.81	0.50	0.41	0.27	81.10	90.96
	Search	1.73	0.75	0.64	0.25
	Write results	5.02	4.06	4.72	3.77
	Sum	19.02	12.61	17.20	11.53	131.65	142.06
GS12878 (3,904,269 lines, 387MB)	Read data	11.65	7.48	11.64	7.41	51.60	51.00
	Sort	0.81	0.51	0.42	0.27	81.70	92.48
	Search	1.75	0.75	0.66	0.25
	Write results	5.07	4.14	4.88	3.91
	Sum	19.28	12.88	17.59	11.84	133.30	143.48
GS12879 (3,909,320 lines, 388MB)	Read data	11.68	7.50	11.73	7.55	51.42	51.00
	Sort	0.81	0.51	0.41	0.27	81.90	92.78
	Search	1.74	0.75	0.65	0.25
	Write results	5.10	4.15	4.78	3.90
	Sum	19.33	12.92	17.57	11.97	133.32	143.78
GS12880 (3,901,545 lines, 387MB)	Read data	11.87	7.41	11.73	7.36	51.16	51.10
	Sort	0.81	0.51	0.41	0.27	82.10	92.43
	Search	1.74	0.75	0.64	0.25
	Write results	5.06	4.15	4.86	4.20
	Sum	19.48	12.83	17.65	12.07	133.26	143.53
GS12881 (3,888,670 linee, 386MB)	Read data	11.63	7.41	11.61	7.40	51.24	51.00
	Sort	0.81	0.51	0.41	0.27	81.80	92.32
	Search	1.74	0.75	0.65	0.25
	Write results	5.11	4.23	5.05	3.87
	Sum	19.28	12.90	17.72	11.78	133.04	143.32
GS12882 (3,898,677 lines, 387MB)	Read data	11.60	7.38	11.60	7.52	51.24	51.10
	Sort	0.81	0.51	0.41	0.27	82.00	92.26
	Search	1.75	0.75	0.65	0.25
	Write results	5.12	4.22	4.80	4.00
	Sum	19.28	12.87	17.47	12.04	133.24	143.36
GS12883 (3,883,072 lines, 385MB)	Read data	11.67	7.48	11.60	7.35	51.13	51.40
	Sort	0.81	0.51	0.41	0.27	81.30	91.66
	Search	1.74	0.75	0.65	0.25
	Write results	5.04	4.10	4.87	3.93
	Sum	19.26	12.84	17.53	11.79	132.43	143.06
GS12884 (3,822,445 lines, 379MB)	Read data	11.58	7.35	11.52	7.26	50.23	51.00
	Sort	0.81	0.50	0.41	0.26	81.30	90.85
	Search	1.72	0.75	0.64	0.24
	Write results	4.98	4.17	4.74	3.83
	Sum	19.09	12.77	17.31	11.59	131.53	141.85
GS12885 (3,801,784 lines, 377MB)	Read data	11.47	7.28	11.52	7.49	49.97	51.20
	Sort	0.81	0.49	0.41	0.25	80.90	90.23
	Search	1.70	0.75	0.63	0.25
	Write results	4.99	4.29	4.73	3.85
	Sum	18.97	12.81	17.29	11.84	130.87	141.43
GS12886 (3,747,040 lines, 371MB)	Read data	11.43	7.34	11.49	7.23	49.20	51.10
	Sort	0.81	0.48	0.41	0.25	80.60	89.28
	Search	1.69	0.75	0.63	0.24
	Write results	4.87	4.01	4.79	3.94
	Sum	18.79	12.58	17.32	11.67	129.80	140.38
GS12887 (3,762,638 lines, 373MB)	Read data	11.46	7.25	11.49	7.32	49.59	51.30
	Sort	0.81	0.49	0.42	0.26	80.60	89.41
	Search	1.68	0.75	0.62	0.24
	Write results	4.92	4.10	4.85	3.96
	Sum	18.88	12.59	17.38	11.78	130.19	140.71
GS12888 (3,866,738 lines, 384MB)	Read data	11.52	7.34	11.54	7.40	50.43	51.10
	Sort	0.81	0.50	0.41	0.27	82.00	91.68
	Search	1.73	0.75	0.64	0.25
	Write results	5.14	4.19	5.12	4.10
	Sum	19.20	12.78	17.72	12.01	132.43	142.78
GS12889 (3,643,266 lines, 361MB)	Read data	11.28	7.13	11.36	7.16	47.88	51.10
	Sort	0.81	0.47	0.41	0.24	80.00	86.88
	Search	1.64	0.75	0.62	0.24
	Write results	4.61	3.98	4.61	3.86
	Sum	18.34	12.32	17.00	11.51	127.88	137.98
GS12890 (3,906,044 lines, 388MB)	Read data	11.70	7.42	11.58	7.56	51.35	51.20
	Sort	0.81	0.51	0.41	0.27	81.80	92.41
	Search	1.75	0.76	0.65	0.25
	Write results	5.19	4.27	4.95	4.04
	Sum	19.44	12.96	17.60	12.11	133.15	143.61
GS12891 (3,879,467 lines, 385MB)	Read data	11.65	7.42	11.67	7.60	50.91	51.30
	Sort	0.81	0.50	0.41	0.26	81.60	91.80
	Search	1.74	0.75	0.65	0.25
	Write results	5.07	4.20	4.91	4.00
	Sum	19.27	12.88	17.64	12.10	132.51	143.10
GS12892 (3,827,308 lines, 380MB)	Read data	11.60	7.37	11.67	7.44	50.54	51.60
	Sort	0.81	0.50	0.41	0.26	81.70	90.60
	Search	1.71	0.75	0.63	0.24
	Write results	4.97	4.20	4.80	4.06
	Sum	19.10	12.82	17.51	12.01	132.24	142.20
GS12893 (3,894,582 lines, 386MB)	Read data	11.67	7.45	11.74	7.39	51.26	51.30
	Sort	0.81	0.51	0.41	0.27	82.20	92.06
	Search	1.74	0.76	0.65	0.25
	Write results	5.14	4.33	4.95	4.28
	Sum	19.37	13.04	17.75	12.19	133.46	143.36
Average (3,845,133 lines, 381MB)	Read data	11.58	7.37	11.59	7.39	50.57	51.17
	Sort	0.81	0.50	0.41	0.26	81.45	91.18
	Search	1.72	0.75	0.64	0.25
	Write results	5.02	4.16	4.85	3.97
	Sum	19.14	12.79	17.49	11.87	132.02	142.35