Enterprise 23.1.x

New features

Launchpad redesign and pipeline enhancements

To enhance pipeline search and navigation capabilities, we now support a new list view to complement the existing card view. The list view allows users to efficiently search for and navigate to their pipeline of choice, while also ensuring that the most relevant information is visible and the relationships between pipelines are clear. With this new feature, users can access their pipelines in either card or list view, making them easier to manage.

We've also introduced a new pipeline detail view that shows in-depth information about each pipeline without needing to access the edit screen.

Enhanced support for Fusion file system

Tower 23.1 introduces support for the Fusion file system in Google Cloud Batch environments. Fusion is a distributed, lightweight file system for cloud-native pipelines that has been shown to improve performance by up to ~2.2x compared to cloud native object storage.

With this new integration, Google Cloud Batch users can enjoy a faster, more efficient, and cheaper processing experience. Fusion offers many benefits, including faster real-time data processing, batch processing, and ETL operations, making it a valuable tool for managing complex data pipelines. By using Fusion with Google Cloud Batch, users can run their data integration workflows directly against data residing in Google Cloud Storage. This integration will allow Google users to streamline their data processing workflows, increase productivity, reduce cloud spending, and achieve better outcomes.

Wave WebSockets support

We have added a new secure way to connect two elements, Tower and Wave, using WebSockets. This is an important addition for our enterprise customers as it ensures connection safety, improved efficiency, and better control over traffic sent between Tower and Wave. This connection will help facilitate the adoption of Fusion by enterprise customers, as it provides a more secure and reliable way to manage their data integration workflows. With WebSockets, users can easily connect their Tower and Wave instances and take advantage of the many benefits that Fusion has to offer.

Other enhancements

• Save executed runs as pipelines
• Improved all runs list view and filtering
• Filter runs by label
• Admin panel enhancements: team and workspace management
• Additional dashboard enhancements:
  • Export dashboard data to CSV
  • Improved date filtering
• Default resource labels for compute environments per workspace
• Fusion log download
• Upgrade Micronaut to 3.8.5
• Tower Agent connection sharing
• Customizable log format
• AWS Parameter store support (distributed config values)
• Azure Repos credential support
• Fusion v2 EBS disk optimized configuration

Breaking changes and warnings

Breaking changes and instructions listed here apply when updating from Tower version 22.4. If you are updating from an earlier version, see the release notes of previous versions for a complete picture of changes that may affect you.

Updated AWS permissions policies

Several new Tower features over the last few releases require updated AWS IAM permissions policies. Retrieve and apply the latest policy files here.

Wave requires container registry credentials

The Wave containers service uses container registry credentials in Tower to authenticate to your (public or private) container registries. This is separate from your existing cloud provider credentials stored in Tower.

This means that, for example, AWS ECR (Elastic Container Registry) authentication requires an ECR container registry credential if you are running a compute environment with Wave enabled, even if your existing AWS credential in Tower has IAM access to your ECR.

See the relevant container registry credentials page for provider-specific instructions.

Upgrade steps

This Tower version requires a database schema update. Follow these steps to update your DB instance and the Tower installation.

!!! warning "" To ensure no data loss, the database volume must be persistent on the local machine. Use the volumes key in the db or redis section of your docker-compose.yml file to specify a local path to the DB or Redis instance.

1. Make a backup of the Tower database.

2. Download and update your container versions.

3. Redeploy the Tower application:

   docker-compose:

   • To migrate the database schema, restart the application with docker-compose down, then docker-compose up.

   kubernetes:

   • Update the cron service with kubectl apply -f tower-cron.yml. This will automatically migrate the database schema.
   • Update the frontend and backend services with kubectl apply -f tower-srv.yml.

   custom deployment:

   • Run the /migrate-db.sh script provided in the backend container. This will migrate the database schema.
   • Deploy Tower following your usual procedures.

Nextflow launcher image

If you must host your nf-launcher container image on a private image registry, copy the nf-launcher image to your private registry. Then update your tower.env with the launch container environment variable:

TOWER_LAUNCH_CONTAINER=<FULL_PATH_TO_YOUR_PRIVATE_IMAGE>

!!! warning "" If you're using AWS Batch, you will need to configure a custom job definition and populate the TOWER_LAUNCH_CONTAINER with the job definition name instead.

Changelog

For a detailed list of all changes, see the Nextflow Tower Changelog.

Sharing feedback

Share your feedback via support.seqera.io.