Deploying FILER

FILER supports installation on a local server or cloud of a full copy of all FILER tracks or of a custom subset of FILER data. For steps/instructions, please see corresponding sections on deploying a full copy of FILER or deploying of a custom subset. Please also refer to the Hardware and Software sections for requirements/prerequisites for a successful installation.

FILER code repository (https://github.com/wanglab-upenn/FILER2) provides the scripts and instructions necessary to deploy all of the FILER or a subset of FILER tracks on a local server or cloud computing instance.

Installing pre-requisites (FILER Giggle and tools)

Linux/macOS/Windows WSL 2 with Homebrew

## install Homebrew

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"


## install all the necessary tools: FILER Giggle, htslib etc

brew install htslib jq miller samtools mawk wget git coreutils pkuksa/tap/filer-giggle

This will install giggle, jq, mlr, mawk, wget binaries.

Installing without Homebrew

Ubuntu/Debian

sudo apt-get install -y tabix jq miller samtools mawk wget git coreutils

CentOS/Fedora

sudo yum install -y htslib jq miller samtools mawk wget git coreutils

Installing FILER Giggle from source


git clone https://github.com/pkuksa/FILER_giggle.git FILER_giggle
cd FILER_giggle
make -f Makefile ## Linux
make -f Makefile.macos ## Macos

Installing command-line FILER scripts


git clone https://github.com/wanglab-upenn/FILER2 FILER2_scripts

Downloading FILER metadata

Track metadata

Full FILER metadata (including download URLs) are available in the templated metadata format (TSV):

Latest GRCh38/hg38 FILER metadata

Latest GRCh37/hg19 FILER metadata

Please refer to the FILER_v2_metadata_schema.xlsx [XLS spreadsheet; June 2026; 14KB] for the description of the metadata fields.

Metadata schema is also available as JSON.

Obtaining FILER metadata on the command line:

wget "https://filer2.niagads.org/metadata/hg38/download/tsv" -O filer2.hg38.metadata.template.tsv
wget "https://filer2.niagads.org/metadata/hg19/download/tsv" -O filer2.hg19.metadata.template.tsv

Columns processed_file_download_url, wget_command contain a track URL and a wget command for downloading and placing the track under the proper directory (data source, assay, data format, genome build hierarchy).

File schemas

Track file schemas: https://filer2.niagads.org/metadata/filer2.schemas.latest.tsv.

Use file schemas file to find description of all file fields for any of FILER tracks: file format column file_format in the track metadata corresponds to the FILER_BED_format column in the schemas file, with the corresponding row storing FILER_BED_type (bedX+Y notation), FILER_BED_schema (semicolon-separated list of fields), FILER_BED_total_columns (total number of fields in the file), and FILER_BED_autoSQL_schema_files (AutoSQL track schema).

Downloading FILER data

Use install_filer.sh script from command-line FILER tools


bash install_filer.sh <target_dir> <full_or_filtered_templated_metadata> <config_file>

install_filer.sh installs FILER tracks specified in the metadata locally on your system. The install_filer.sh script will first download all the tracks specified in the templated metadata file into the target directory and then Giggle-index them. Note that the templated metadata can be additionally custom-filtered/subsetted to include only the tracks of interest for download/installation (see below).

Bulk download tracks of interest

For example, to download Brain ATAC-seq tracks:


wget "https://filer2.niagads.org/search?query=ATAC-seq Brain&outputFormat=tsv" -O filer_metadata.selected.tsv
bash install_filer.sh FILER2_data filer_metadata.selected.tsv data/filer.homebrew.ini

This example uses filer.homebrew.ini config provided in the FILER2_scripts Github repository (https://github.com/wanglab-upenn/FILER2/blob/master/data/filer.homebrew.ini). This config is for Homebrew-based installations, if you are using custom/non-Homebrew installation, please adjust filepaths accordingly to match your system's setup.

Tracks of interest can also be obtained by filtering the full FILER metadata. E.g., to get all the enhancer-related tracks:


wget "https://filer2.niagads.org/metadata/hg38/download/tsv" -O filer2.hg38.metadata.template.tsv
grep -i enhancers filer2.hg38.metadata.template.tsv > filer2.hg38.metadata.enhancers.template.tsv
bash install_filer.sh FILER2_data filer2.hg38.metadata.enhancers.tsv data/filer.homebrew.ini

Downloading individual data sources in FILER

Any individual data source can be downloaded using dataSource parameter:


wget "https://filer2.niagads.org/metadata/hg38/download/tsv?dataSource=MiGA" -O filer2.hg38.MiGA.tsv
bash install_filer.sh FILER2_data filer2.hg38.MiGA.tsv data/filer.homebrew.ini

Deploying a full copy of FILER


wget "https://filer2.niagads.org/metadata/hg38/download/tsv" -O filer2.hg38.metadata.template.tsv
bash install_filer.sh FILER2_data filer2.hg38.metadata.template.tsv data/filer.homebrew.ini

This will download/install all hg38 tracks in FILER. NOTE: this will require a significant amount of storage.

To check the expected download size:


awk 'BEGIN{FS=OFS="\t"}{if (FNR==1) next; total+=$18}END{print (total/10^9 "GB")}' filer2.hg38.metadata.template.tsv
4176.14GB

Giggle indexes will additionally take about the amount of storage.

Advanced download by data collection

Individual FILER data collections can be also accessed/downloaded using a data collection table.

Querying FILER data


bash FILER2_scripts/data_querying/get_overlaps.sh --inBed FILER2_scripts/data/test.20intervals.bed.gz \
--configFile FILER2_scripts/data/filer.homebrew.ini \
--outputDir filer_test_overlaps/ \
--verboseSearch 1 \
--genomeBuild hg38 \
--forceOverwrite 1