Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: CI

on:
push:
branches: [master]
tags: ['*']
pull_request:

jobs:
test:
name: Julia ${{ matrix.version }} - ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
version: ['1.10', '1.12']
os: [ubuntu-latest, macos-latest]
steps:
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: ${{ matrix.version }}
- uses: julia-actions/cache@v2
# Manifest.toml is generated with Julia 1.12, so older versions resolve their own within the [compat] bounds
- name: Remove Manifest.toml for older Julia versions
if: matrix.version != '1.12'
run: rm Manifest.toml
- uses: julia-actions/julia-buildpkg@v1
- uses: julia-actions/julia-runtest@v1
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# CHANGELOG

## v1.10.0

* BUGFIX - `--exclude_regions_file` crashed with "Cannot `convert` an object of type String to an object of type Char" unless `--stranded` was "yes" or "reverse". Added regression tests for both stranded and unstranded runs.
* FADU is now a proper Julia package (module `FADU`, code in src/, Project.toml and Manifest.toml at the repository root). `julia fadu.jl <options>` still works the same way.
* FADU's own code is now precompiled (using PrecompileTools and the tiny test dataset as the workload), so runs start about 3x faster than v1.9.2.
* Adding `Pkg.test()` support (test/runtests.jl) and GitHub Actions CI on Julia v1.10 and v1.12.
* Adding an experimental `fadu` command for Julia v1.12+ via `Pkg.Apps.add`, and support for `julia -m FADU <options>`.
* Renamed FADU's internal BAM overlap iterator to `eachalignmentoverlap`. It previously overwrote XAM's `GenomicFeatures.eachoverlap` method, which is not allowed in a precompiled package. Output is unchanged.
* Moved the tiny regression dataset from test_data/tiny to test/data and the test script to test/run_test.sh.
* Docker: the container's working directory is now /data (mount your files there) instead of /opt/FADU.

## v1.9.2

* Raising minimum Julia version to v1.10 and regenerating fadu_pkgs/Manifest.toml under Julia v1.12. Dockerfile now uses the julia:1.12 image.
Expand Down
27 changes: 12 additions & 15 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,19 @@
FROM julia:1.12
LABEL maintainer="Shaun Adkins (sadkins@som.umaryland.edu)"

RUN mkdir -p /opt/FADU

WORKDIR /opt/FADU

COPY ./alignment_overlaps.jl /opt/FADU
COPY ./bam_record.jl /opt/FADU/
COPY ./fadu.jl /opt/FADU/
COPY ./feature_counts.jl /opt/FADU/
COPY ./gff_feature.jl /opt/FADU/
COPY ./Project.toml ./Manifest.toml ./fadu.jl /opt/FADU/
COPY ./src /opt/FADU/src
# The tiny test dataset is used as the precompilation workload
COPY ./test /opt/FADU/test

# Install and precompile FADU and its dependencies into the image
ENV JULIA_DEPOT_PATH=/opt/julia_depot
ENV JULIA_PROJECT=/opt/FADU
RUN julia -e 'using Pkg; Pkg.instantiate(); Pkg.precompile()'

# force compilation of external, shared packages copied in from Manifest.toml and Project.toml
# For some reason these would not copy to the "root" home directory
RUN mkdir -p /tmp/.julia/environments/v1.12
COPY ./fadu_pkgs/Manifest.toml /tmp/.julia/environments/v1.12/Manifest.toml
COPY ./fadu_pkgs/Project.toml /tmp/.julia/environments/v1.12/Project.toml
ENV JULIA_DEPOT_PATH=/tmp/.julia
RUN julia -e 'using Pkg; Pkg.precompile(); Pkg.instantiate()' && julia -e 'using BGZFStreams, GenomicFeatures, GFF3, Indexes, XAM, BED'
# Input and output files are expected to be mounted here
WORKDIR /data

ENTRYPOINT ["julia", "/opt/FADU/fadu.jl"]
ENTRYPOINT ["julia", "/opt/FADU/fadu.jl"]
12 changes: 9 additions & 3 deletions fadu_pkgs/Manifest.toml → Manifest.toml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 15 additions & 0 deletions NOTES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Notes for package author

## Updating the Manifest.toml file

Manifest.toml pins the dependency versions used for releases, Docker and bioconda. Regenerate it from the repository root with the Julia version you want to target (currently v1.12):

1. `rm Manifest.toml` - This needs to be removed so the next step will resolve a new one
2. `julia --project=. -e 'using Pkg; Pkg.resolve(); Pkg.instantiate(); Pkg.precompile()'`
3. Run `julia --project=. -e 'using Pkg; Pkg.test()'` to make sure the output has not changed

## Making a release

1. Bump `version` in Project.toml and `VERSION_NUMBER` in src/FADU.jl (a test fails if they differ)
2. Add a CHANGELOG.md entry and update "Current FADU release" in README.md and docs/index.md
3. Commit, then tag with `git tag -a vX.Y.Z -m "vX.Y.Z"` and push the tag
13 changes: 13 additions & 0 deletions fadu_pkgs/Project.toml → Project.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
name = "FADU"
uuid = "866296ad-388f-4871-8691-d7485d094d7e"
version = "1.10.0"
authors = ["adkinsrs <sadkins@som.umaryland.edu>"]

[deps]
Expand All @@ -8,6 +11,7 @@ GFF3 = "af1dc308-cb6b-11e8-32f0-31192efa90f6"
GenomicFeatures = "899a7d2d-5c61-547b-bef9-6698a8d05446"
Indexes = "4ffb77ac-cb80-11e8-1b35-4b78cc642f6d"
Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"
StructArrays = "09ab397b-f2b6-538f-b94a-2f83cf4a842a"
XAM = "d759349c-bcba-11e9-07c2-5b90f8f05f7c"
Expand All @@ -19,6 +23,15 @@ BGZFStreams = "0.3.2"
GFF3 = "0.2.3"
GenomicFeatures = "2.1.0"
Indexes = "0.1.3"
PrecompileTools = "1"
StructArrays = "0.6.17"
XAM = "0.3.1"
julia = "1.10"

[apps.fadu]

[extras]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["Test"]
36 changes: 28 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,15 @@ Most current available quantification tools for transcriptomics analyses have be

## Current FADU release

v1.9.2
v1.10.0

## Requirements

Julia - v1.10 or later (fadu_pkgs/Manifest.toml was generated with Julia v1.12)
Julia - v1.10 or later (Manifest.toml was generated with Julia v1.12)

### Packages

NOTE: Packages installation instructions can be found at https://docs.julialang.org/en/v1.4/stdlib/Pkg/index.html#Getting-Started-1
FADU is a Julia package, so these are installed for you by `Pkg.instantiate()` (see "Installation instructions" below).

* ArgParse - v1
* Logging
Expand All @@ -36,6 +36,7 @@ NOTE: Packages installation instructions can be found at https://docs.julialang.
* StructArrays.jl - v0.6.17
* XAM.jl - v0.3.1
* BED.jl - v0.3.0
* PrecompileTools.jl - v1

### NOTE on versions

Expand Down Expand Up @@ -76,8 +77,21 @@ First `cd` to the directory of choice to install FADU
```bash
git clone https://github.com/IGS/FADU.git
cd FADU
julia --project=. -e 'using Pkg; Pkg.instantiate()'
```

The last step downloads FADU's dependencies and precompiles FADU, which takes a minute or two the first time. After that, run FADU with `julia ./fadu.jl <options>` from anywhere (use the full path to fadu.jl if you are not in the FADU directory).

### Installing a `fadu` command (Julia v1.12+, experimental)

Julia v1.12 can install FADU as a command-line app:

```bash
julia -e 'using Pkg; Pkg.Apps.add(url="https://github.com/IGS/FADU.git")'
```

This puts a `fadu` executable in `~/.julia/bin` (add that directory to your PATH), which takes the same options as `julia ./fadu.jl`. Julia's app support is still experimental, so the clone-based install above is the recommended route.

## Example command

For this example we will create a directory called "fadu\_output" to write output into. This example uses a BAM alignment file of subsampled Wolbachia reads. This example should take about 60 seconds to run.
Expand Down Expand Up @@ -106,13 +120,19 @@ cds1005 816 0.0 0.00 0.00

## Testing

A small regression dataset (a ~20 kb slice of an E. coli paired-end RNA-Seq BAM, plus the matching GFF3 annotation) lives in "test_data/tiny". To run FADU on it and compare against the expected output:
A small regression dataset (a ~20 kb slice of an E. coli paired-end RNA-Seq BAM, plus the matching GFF3 annotation) lives in "test/data". To run the test suite:

```bash
julia --project=. -e 'using Pkg; Pkg.test()'
```

To check only the command-line output against the expected counts, without Julia's test framework:

```bash
bash ./test_data/tiny/run_test.sh
bash ./test/run_test.sh
```

By default the script runs `julia --project=./fadu_pkgs ./fadu.jl`. Set the `FADU_CMD` environment variable to test a different FADU command (e.g. an installed copy).
By default the script runs `julia ./fadu.jl`. Set the `FADU_CMD` environment variable to test a different FADU command (e.g. an installed copy).

## Usage

Expand Down Expand Up @@ -218,10 +238,10 @@ docker build -t fadu .
This Docker image uses Julia 1.12 and installs the various packages required to run FADU. Once the image has been built you can create and run a FADU container by running the following:

```bash
docker run -v $PWD:/opt/FADU fadu <fadu_options>
docker run -v $PWD:/data fadu <fadu_options>
```

This command assumes that your input files are in your current working directory and that your output will be written to a space relative to your current working directory as well.
This command mounts your current working directory at /data, which is the container's working directory, so input and output paths relative to your current working directory work as-is.

## Issues

Expand Down
36 changes: 28 additions & 8 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,15 @@ Most current available quantification tools for transcriptomics analyses have be

## Current FADU release

v1.9.2
v1.10.0

## Requirements

Julia - v1.10 or later (fadu_pkgs/Manifest.toml was generated with Julia v1.12)
Julia - v1.10 or later (Manifest.toml was generated with Julia v1.12)

### Packages

NOTE: Packages installation instructions can be found at https://docs.julialang.org/en/v1.4/stdlib/Pkg/index.html#Getting-Started-1
FADU is a Julia package, so these are installed for you by `Pkg.instantiate()` (see "Installation instructions" below).

* ArgParse - v1
* Logging
Expand All @@ -36,6 +36,7 @@ NOTE: Packages installation instructions can be found at https://docs.julialang.
* StructArrays.jl - v0.6.17
* XAM.jl - v0.3.1
* BED.jl - v0.3.0
* PrecompileTools.jl - v1

### NOTE on versions

Expand Down Expand Up @@ -76,8 +77,21 @@ First `cd` to the directory of choice to install FADU
```bash
git clone https://github.com/IGS/FADU.git
cd FADU
julia --project=. -e 'using Pkg; Pkg.instantiate()'
```

The last step downloads FADU's dependencies and precompiles FADU, which takes a minute or two the first time. After that, run FADU with `julia ./fadu.jl <options>` from anywhere (use the full path to fadu.jl if you are not in the FADU directory).

### Installing a `fadu` command (Julia v1.12+, experimental)

Julia v1.12 can install FADU as a command-line app:

```bash
julia -e 'using Pkg; Pkg.Apps.add(url="https://github.com/IGS/FADU.git")'
```

This puts a `fadu` executable in `~/.julia/bin` (add that directory to your PATH), which takes the same options as `julia ./fadu.jl`. Julia's app support is still experimental, so the clone-based install above is the recommended route.

## Example command

For this example we will create a directory called "fadu\_output" to write output into. This example uses a BAM alignment file of subsampled Wolbachia reads. This example should take about 60 seconds to run.
Expand Down Expand Up @@ -107,13 +121,19 @@ cds1005 816 0.0 0.00 0.00

## Testing

A small regression dataset (a ~20 kb slice of an E. coli paired-end RNA-Seq BAM, plus the matching GFF3 annotation) lives in "test_data/tiny". To run FADU on it and compare against the expected output:
A small regression dataset (a ~20 kb slice of an E. coli paired-end RNA-Seq BAM, plus the matching GFF3 annotation) lives in "test/data". To run the test suite:

```bash
julia --project=. -e 'using Pkg; Pkg.test()'
```

To check only the command-line output against the expected counts, without Julia's test framework:

```bash
bash ./test_data/tiny/run_test.sh
bash ./test/run_test.sh
```

By default the script runs `julia --project=./fadu_pkgs ./fadu.jl`. Set the `FADU_CMD` environment variable to test a different FADU command (e.g. an installed copy).
By default the script runs `julia ./fadu.jl`. Set the `FADU_CMD` environment variable to test a different FADU command (e.g. an installed copy).

## Usage

Expand Down Expand Up @@ -219,10 +239,10 @@ docker build -t fadu .
This Docker image uses Julia 1.12 and installs the various packages required to run FADU. Once the image has been built you can create and run a FADU container by running the following:

```bash
docker run -v $PWD:/opt/FADU fadu <fadu_options>
docker run -v $PWD:/data fadu <fadu_options>
```

This command assumes that your input files are in your current working directory and that your output will be written to a space relative to your current working directory as well.
This command mounts your current working directory at /data, which is the container's working directory, so input and output paths relative to your current working directory work as-is.

## Issues

Expand Down
Loading
Loading