Skip to content

HGDP + 1000 Genomes Joint Callset

Resource Location

/labs/SysMedBio/Q/Resources/Genomics/HGDP_1KG

/Volumes/lab/SysMedBio/Resources/Genomics/HGDP_1KG

Q:\SysMedBio\Resources\Genomics\HGDP_1KG

This resource contains genotype data, metadata, and scripts for the harmonized, deeply sequenced Human Genome Diversity Project (HGDP) and 1000 Genomes Project (1kGP) joint callset.

Scientific Context

Component Projects

  • The 1000 Genomes Project (1kGP): A pioneer resource cataloging common genetic variations in human populations across 26 populations from 5 global regions (Africa, East Asia, South Asia, Europe, and the Americas).
  • The Human Genome Diversity Project (HGDP): A collection of DNA samples representing global genetic diversity, including many historically underrepresented or isolated indigenous populations (covering 54 populations across 7 regions, including Oceania, the Middle East, Central/South Asia, and the Americas).

Joint Calling & Harmonization

Historically, the HGDP and 1kGP datasets were sequenced using different platforms, aligned to different reference assemblies (GRCh37 vs. GRCh38), and processed using different variant-calling pipelines. Direct combination introduced severe technical batch effects that confounded population genetics and ancestry analyses.

The Genome Aggregation Database (gnomAD) consortium resolved this by re-aligning and re-calling raw sequences from both projects using a unified pipeline (GRCh38 assembly, GATK HaplotypeCaller). - Dataset Scale: 4,094 individuals from approximately 80 global populations sequenced to deep coverage (mean > 30x). - Variants: Over 153 million high-quality SNPs, indels, and structural variants. - Utility: A globally accessible, unrestricted reference panel used heavily for ancestry estimation (PCA, ADMIXTURE), haplotype phasing, and genotype imputation.

Main Publications

  • Journal Publication: Koenig, Z., Yohannes, M. T., Nkambule, L. L., et al. (2024). "A harmonized public resource of deeply sequenced diverse human genomes." Genome Research, 34(5), 796–809.
  • DOI: 10.1101/gr.278378.123

Available Files

The resource folder contains raw genotype files, processed PLINK2 datasets, sample metadata, and scripts representing a complete genetic ancestry pipeline:

Setup & Metadata

  • HGDP_1KG_gnomad_meta_updated.tsv: Full sample metadata from gnomAD, including population information.
  • HGDP_1KG_gnomad_ancestry.tsv: A simplified ancestry reference mapping individual samples (IID) to continental ancestries (Ancestry).
  • HGDP_1KG_post_qc_summary.tsv: Post-QC summary table listing population sizes, coverage, and variant counts.
  • 01_Download_vcf_files.slurm: SLURM script to download chromosome VCFs from gnomAD and convert to PLINK2.
  • 02_Merge_pgen_files.slurm: SLURM script to merge chromosome PLINK2 files into a single genome-wide dataset.
  • 03_Final_variant_trimming.slurm: SLURM script performing quality and frequency filtering on the merged dataset.

Genotype Datasets (plink2/)

  • gnomad.genomes.v3.1.2.hgdp_tgp.final.[pgen|psam|pvar.zst]: Trimmed joint genotype callset in PLINK2 binary format.
  • gnomad.genomes.v3.1.2.hgdp_tgp.simplified.[pgen|psam|pvar]: A simplified version where INFO fields are stripped and variant IDs standardized (chr@:#:$r:$a) to reduce storage space.

Analysis & Orchestration (scripts/)

  • run_ancestry_pipeline.R: An R script orchestrating sample extraction, linkage disequilibrium (LD) pruning, kinship filtering (via KING), PCA, ADMIXTURE cross-validation (K=2 to 14), and ancestry modeling (supervised K=5).

Results and Figures (output/)

  • Quality Control (output/QC/):
    • high_quality_samples.txt: IDs of samples flagged as high quality in the metadata.
    • pruned_snps.prune.in & pruned_snps.prune.out: Variant lists indicating SNPs included or excluded during Linkage Disequilibrium (LD) pruning.
    • unrelated_samples.king.cutoff.in.id & unrelated_samples.king.cutoff.out.id: RET/REM lists from kinship filtering.
    • clean_reference_samples.txt: Retained unrelated, high-quality, non-admixed reference individuals.
  • Principal Component Analysis (output/PCA/):
    • initial_pca.eigenval & initial_pca.eigenvec: Calculated eigenvalues and eigenvectors.
  • ADMIXTURE (output/Admixture/):
    • admixture: ADMIXTURE binary executable.
    • admixture_input.[bed|bim|fam]: Input files.
    • admixture_input.pop: Target labels mapping reference panel samples.
    • admixture_input.<K>.[P|Q]: Frequency and proportion outputs for K configurations.
    • cv_errors.tsv: ADMIXTURE cross-validation errors for K=2 to 14.
  • Plots (output/plots/):
    • admixture_cv_error.png: Plot illustrating cross-validation error across cluster counts.
    • initial_pca_pc1_pc2.png: PC1 vs PC2 scatter plot, colored by ancestry.
    • admixture_proportions_k5.png: Stacked bar plot showing ancestry proportions across individuals.