IDA User Guide

Hate reading manuals? Too busy to read the full user guide? Try the IDA Quick Start Guide

Overview

Welcome to the IDA service!

IDA (ida.fairdata.fi) is a research data storage service organized by the Finnish Ministry of Education and Culture and produced by CSC — IT Center for Science.

The service is intended for stable research data, both raw data and processed data, and is offered free of charge to users affiliated with Finnish universities and polytechnics, Finnish research institutes and research funded by the Academy of Finland.

This guide provides an overview of the features and functions of the service, organized by major functional area and view. If you are new to the service, it is recommended that you read this guide fully before using the service. The images below give you an overview of IDA’s browser user interface. You can also use IDA with command line tools.

Project Data Storage

Each project in IDA has an allocated quota of storage space. This storage space is divided into two distinct areas: a Staging area (read/write) and a Frozen area (read-only). Both areas share the same project quota, so there is no need to manage or reserve space for each use separately.

All project members have equal rights to add, remove, and operate on project data stored in the IDA service.

Users of the service may belong to one or more projects with which they collaborate.

In both areas project members can:

• view directories, files and file sizes
• download files and directories to their own computer
• share files and directories outside the service via a temporary shared link
• delete files and directories

The Staging area is intended for collecting and organizing research data in preparation for storage in the frozen area.

In the staging area:

• files and folders can be added, rearranged, renamed, overwritten, and removed
• file metadata are not saved in the metadata repository nor visible to other services
• files are not replicated, so storing in the staging area doesn’t prevent data loss in case of a major hardware failure (single disk failure will not cause data loss)

The Frozen area is intended for storing stable research data in an immutable state.

In the frozen area:

• files are given unique identifiers, checksums are generated, and file metadata stored in a metadata repository
• files are replicated on another media to prevent data loss in case of a major hardware failure
• files are visible and accessible to other Fairdata services, such as for Qvain metadata tool and for the Digital Preservation Service for Research Data

Data should be moved to the frozen area once no further changes are expected to be made. Data in the frozen area is available to other Fairdata services and they can be included as a part of user-defined datasets using the Qvain metadata tool.

All project members can move data from the staging area to the frozen area (as described below). Because of the procedures for data preservation, data in the frozen area can not be re-arranged, renamed or overwritten. Frozen data can moved back to the staging area or deleted entirely from the service, but users should be aware of the effects of these actions. If data is moved back to the staging area or deleted, the unique file identifiers and file metadata will be discarded. This means that any references to those files within any defined datasets and download links to data will become permanently invalid.

Please note that IDA’s frozen area is not the same as the Digital Preservation Service. The Digital Preservation Service for Research Data (Fairdata-PAS) is a separate service. IDA and Qvain can be utilized when transferring data and related metadata to Fairdata PAS. More information about the Fairdata service integrations: fairdata.fi/en/services/components/

Monitoring and Adjusting Project Quota

To monitor the use of IDA storage space for a project go to the MyCSC customer portal and select your project from the “My Projects” menu after logging in. You will see the total amount of storage space for the project, the total amount of stored data, and the amount of frozen data under the heading “IDA storage usage”. If you belong to just one project in IDA, the used quota is available in the browser UI (ida.fairdata.fi) in the root view, under the frozen and staging folder.

The Project Manager can adjust the project’s IDA quota in MyCSC customer portal. To change the project quota, log in MyCSC and select your project from “My Projects” menu. In the project’s “Services” menu, click “IDA” and then click on “Configuration”. Fill in the new quota in GiB and click on “Apply”. Your quota granter organisation will handle requests to increase the quota. A request to decrease the quota will be accepted automatically.

Please note that it is only possible to adjust the IDA quota the day after the IDA storage space is is first applied to the CSC project.

Recommended Browsers

We recommend using the latest stable versions of either the Firefox, Chrome, or Edge web browsers for interacting with the IDA service web UI. We advise against using Safari due to issues when uploading files, such that hidden files or folders (i.e. files or folders beginning with a period ‘.’) cannot be uploaded to IDA reliably via the Safari browser.

In addition, when uploading data from a Mac OS-X machine, even if using Firefox or Chrome to interact with the IDA web UI, we advise using the IDA Command Line Tools with an ignore file which excludes ‘.DS_Store’ directories. See https://github.com/CSCfi/ida2-command-line-tools#ignore-file

Getting started

Logging in

Before you can log in, you need to apply for IDA storage space, or join an existing IDA project.

After this, you may log in to the IDA service using either the Haka authentication service, Virtu authentication service or using your CSC user account. To log in, navigate to ida.fairdata.fi.

IDA, Qvain and Etsin use a common single sign-on/sign-off (SSO) service. This means that you will log in to all of these Fairdata services when logging in once through IDA, Qvain or Etsin. Similarly, logging out will log you out of all of the above services.

To log in to the IDA service, click on the “Login” button in the top right corner of the IDA home page. On the login page presented, select either “CSC Login”, “Haka Login” or “VIRTU” and follow the instructions provided.

Note that personal CSC user accounts are created and managed via MyCSC, including changing or resetting your CSC account password.

Upon successfully logging in to the IDA service, you will be presented with the top level files view, described below.

Logging out

To log out from the IDA service, click on the gear icon located in the top right corner of the view, which will open a pull-down menu, and select the “Logout” menu option. On the logout page presented, click the “Logout” button. Logging out will log you out of all of the Fairdata services.

Personal settings

To change the personal settings of the web UI, click on the gear icon located in the top right corner of the view, which will open a pull-down menu, and select the “Settings” menu option.

To change the language, you must log out of the IDA service, and on the IDA home page select another language from the top right corner of the page. The selected language will be used for all Fairdata services and will be remembered until you change it again, either on the IDA home page or in another Fairdata service. The locale used in IDA is derived from your language selection.

Files View

When you first log in to the IDA service, you will be presented with a file browser view showing the staging and frozen folders for all projects to which you have access. The name of the frozen folder for a project will correspond to the project name. The name of the staging folder for a project will correspond to the project name, followed by a plus “+” character. The plus character in the staging folder name serves as a reminder that new data is added to the staging area of the project.

The contents of a folder can be viewed by clicking on the name of the folder in the listing.

To navigate back to a parent folder, either use your web browser’s “Back” functionality, or click on the desired ancestor folder in the pathname above the listing.

To return to the top root of the files listing from a subfolder, click on the home icon at the far left of the pathname above the listing.

To return to the files view from any other view, click on the “Files” at the top of the view.

Staging Area

New files are uploaded to the staging area of a project, which is fully contained within the staging folder of the project.

Please note that files do not need to be packaged before they’re transferred to the service and the files can be published directly as individual files or in folder structures.

Files can be uploaded directly into the root of the staging folder; however, it is strongly recommended that files be organized by subfolders within the staging folder.

Organizing files by e.g. time period and other logical structures can help make management of large numbers of data files easier, and freezing of data easier and more efficient, by selecting and freezing all files within a folder (see below about freezing files).

Learn more about best practices in naming and and structuring your data in CSC Docs. We recommend using common, preferably open file formats. More information about files and file formats in CSC Docs.

NOTE: Files named .htaccess and files with a suffix of either .part or .filepart may not be uploaded. The filenames will need to be changed in some manner, such as zipping the file or adding some other suffix or name change. These restrictions are due to security and other internal management constraints of the underlying Nextcloud platform employed by the IDA service.

Creating Folders

To create a new folder, click on the “+” icon at the top of the view, to the right of the current folder path, and select the option “New folder”. Enter a name for the new folder and press “Return”.

Uploading Individual Files

To upload an individual file to the current folder, click on the “+” icon at the top of the view, to the right of the current folder path, and select the option “Upload file”, and using the file picker presented, navigate to and select a file on your local filesystem.

Uploading Folders and Multiple Files using Drag-and-Drop

It is possible to upload entire folders (and their subfolders) and/or multiple files at the same time by using drag-and-drop.

First navigate in the files view to the folder to which you wish to upload the data, creating new subfolders as required.

Then, open your local filesystem browser, select the desired folders and/or files to upload, and drag-and-drop them into the web browser window where the IDA service is open.

The selected folders and files will then be uploaded to the current folder in the files view, preserving the folder organization as stored on the local filesystem.

Moving or Copying Files or Folders

Files and folders within the staging area of a project may be freely moved or copied elsewhere in the staging area. It is also possible to copy frozen files and folders from the frozen area to the staging area.

To move or copy a file or folder, navigate in the files view to where the file or folder is located and click on the ellipsis “…” menu to the right of the file or folder name, and select the option “Move or copy”.

In the pop-up window presented, navigate to the desired target folder within the staging area and click either “Copy” or “Move”. To create a new target folder, click the plus “+” icon at the top of the pop-up window and enter the name of the new target folder.

Renaming Files or Folders

Files and folders within the staging area of a project may be freely renamed.

To rename a file or folder, navigate in the files view to where the file or folder is located and click on the ellipsis “…” menu to the right of the file or folder name, and select the option “Rename”.

Edit the name as desired and press “Return”.

Downloading Files or Folders

Both individual files and entire folders can be easily downloaded from the IDA service. When downloading folders, the contents of the folder will be packaged for downloading as a single packaged archive file, which can then be unpacked after download.

To download a file or folder, navigate in the files view to where the file or folder is located and click on the ellipsis “…” menu to the right of the file or folder name, and select the option “Download”.

Follow your browser’s instructions regarding how and where to save the file on your local machine.

Deleting Files or Folders

Files and folders within the staging area of a project may be freely deleted.

To delete a file or folder (and all of its content), navigate in the files view to where the file or folder is located and click on the ellipsis “…” menu to the right of the file or folder name, and select the option “Delete”.

In the confirmation dialog presented, click “OK”.

WARNING: The delete action cannot be undone and the deleted data will be permanently and irretrievably removed from the IDA service!

Freezing Files

When files reach sufficient maturity such that they are considered final and should no longer change, they may be frozen, as described in detail below, which will render them immutable and also make them visible to you and your project members in Qvain and Metax API, so that the files can be included in a published dataset. Freezing data doesn’t make them visible to people outside your project.

Both individual files and entire folders can be selected for freezing. When selecting a folder, all files within that folder, and any subfolders, will be frozen.

Freezing will move each selected file to the frozen area, preserving its relative pathname within the staging area, and making them immutable. The freeze action cannot be terminated before it is complete. Depending on the size and number of files being frozen, the background operations may take several hours. The status of the pending freeze action can be viewed in the pending actions listing as described below.

Frozen files are replicated to separate physical storage to guard against loss of data due to hardware failure. Replication employs checksum validation to ensure the integrity of the frozen file copies. If frozen files are later unfrozen or deleted, all replicated copies will be removed.

It is strongly recommended that files are organized by subfolders within the staging area before they are frozen. You are able to freeze a folder that contains maximum of 5000 files, but having less files in a folder makes it easier to browse and download the data. You can organize files by e.g. time period and other logical structures. Learn more about best practices in naming and and structuring your data in CSC Docs.

How to freeze files

To freeze a file or all files within the scope of a folder, navigate in the files view to the location in the staging area of the file or folder and click on the snowflake icon to the right of the file or folder name, which will open the action tab on the right side of the view. Then, in the action tab, click on the button “Freeze”.

In the confirmation dialog presented, if you are absolutely sure you wish to freeze the selected file(s), click “Yes”.

Once the freeze action has been successfully initiated, you will be presented with the option to remain in the file view where you initiated the action, open the file view in the frozen area where the newly frozen files are now located, or by clicking on the pending action link shown in the dialog, you may open the action summary (described in greater detail below).

Note that the freeze action will be rejected if any pathname of any of the files to be frozen conflicts with either an already frozen file, or a file belonging to a pending action. This is to ensure against corruption of already frozen (immutable) files, and their associated metadata and identifiers.

If you wish to freeze a file which shares the same relative pathname as a frozen file, you must either rename the newer file or move it to a different location (pathname) within the staging area, or you must unfreeze or delete the previously frozen file (described further below).

If you wish to freeze a file which shares the same relative pathname as a file in a pending action, you must first wait for the pending action to be completed and then try freezing the file again.

Freezing new files when the parent folder already exists in the frozen area

When a file, or folder, is frozen, any necessary parent folders for each file will be created, as required. If those parent or ancestor folders already exist, that is OK. The relative pathname of each file being frozen will be preserved and created as required, as the file is moved from the staging area to the frozen area. As long as no file exists with the same relative pathname, the freeze operation will proceed as requested, creating all necessary parent or ancestor folders that do not already exist.

Example 1:

You have a folder in the staging area with the relative pathname /EXP1, containing several unfrozen files. You also have a folder in the frozen area with the same relative pathname /EXP1, containing several frozen files. You decide to freeze a file in the staging area with the relative pathname /EXP1/DATA_20180611.csv. The freezing process will verify that no file already exists in the frozen area with the same relative pathname, and will then move the specified file from the staging area to the frozen area. Because the folder /EXP1 already existed in the frozen area, there was no need to create it, nor was there any affect on the already frozen files located in that folder. The folder /EXP1 in the staging area, along with all of the files which have not yet been frozen, remain unchanged in the staging area.

Example 2:

You have a folder in the staging area with the relative pathname /EXP2, containing several files. There is no such folder in the frozen area. You decide to freeze a file in the staging area with the relative pathname /EXP2/DATA_20180922.csv. The freezing process will verify that no file already exists in the frozen area with the same relative pathname, and will then create the folder /EXP2 in the frozen area and move the specified file from the staging area to the frozen area. Because the folder /EXP2 did not already exist in the frozen area it was created. The folder /EXP2 in the staging area, along with all of the files which have not yet been frozen, remain unchanged in the staging area.

Frozen Area

Frozen files are located in the frozen area of a project, which is fully contained within the frozen folder of the project.

Frozen files are immutable, and may not be freely updated, renamed, moved, or otherwise modified the same as files in the staging area. All that may be done to frozen files is to download them, unfreeze them, or delete them entirely from the service; where the latter two options will result in their identity and metadata becoming invalid and no longer available to other data preservation services.

NOTE: Files are not stored persistently in the IDA service until they are frozen.

Frozen File Details

Frozen files are assigned unique identifiers, have their checksums generated, their details stored as metadata in a centralized repository (Fairdata Metax) and they are replicated to separate physical storage.

To view the details of a particular file, navigate in the files view to where the file is located and click on the snowflake icon to the right of the file name, which will open the action tab on the right side of the view. The file details will be shown at the top of the action tab.

Note that if the file is part of a pending action, not all details will be visible, as they may not have been generated yet by the ongoing background processing.

Checksums for frozen files are generated automatically by the service, using the SHA256 algorithm, and are displayed as “sha256:” URIs in hexidecimal form.

After downloading a frozen file, a local checksum can be generated for comparison with the checksum displayed, using the following command:

 sha256sum filename | awk '{print $1}' | tr '[A-Z]' '[a-z]'

Unfreezing Frozen Files

If one or more frozen files need to be modified in any way, including updating, renaming, or moving, it is possible to unfreeze both individial files as well as all files within the scope of a particular folder.

Unfreezing will move each selected file back to the staging area, preserving its relative pathname within the frozen area, and making them again fully editable. The unfreeze action cannot be terminated before it is complete. Depending on the size and number of files being unfrozen, the background operations may take several hours. The status of the pending unfreeze action can be viewed in the pending actions listing as described below.

WARNING: Unfrozen files will no longer be accessible to other services. Unfreezing makes all external references to files invalid. If the files are linked to datasets defined in Qvain, unfreezing deprecates these datasets permanently. A warning dialog will be presented if any datasets would be deprecated by the unfreeze action. If frozen files are unfrozen, all replicated copies will be removed. The unfreeze action, once initiated, cannot be undone!

To unfreeze a file or all files within the scope of a folder, navigate in the files view to the location in the frozen area of the file or folder and click on the snowflake icon to the right of the file or folder name, which will open the action tab on the right side of the view. Then, in the action tab, click on the button “Unfreeze”.

In the confirmation dialog presented, if you are absolutely sure you wish to unfreeze the selected file(s), click “Yes”.

Once the unfreeze action has been successfully initiated, you will be presented with the option to remain in the file view where you initiated the action, open the file view in the staging area where the newly unfrozen files are now located, or by clicking on the pending action link shown in the dialog, you may open the action summary (described in greater detail below).

Note that the unfreeze action will be rejected if any pathname of any of the files to be unfrozen conflicts with either any file in the staging area, or a file belonging to a pending action. This is to ensure against corruption of any files files in staging, and to permit all essential background processing to finish before like named files are refrozen, to avoid potential confusion or disruption in other services which may be utilizing frozen file metadata in some manner.

If you wish to unfreeze a file which shares the same relative pathname as a file in staging, you must either rename the file in staging or move it to a different location (pathname) within the staging area, or you must delete the file in staging.

If you wish to unfreeze a file which shares the same relative pathname as a file in a pending action, you must first wait for the pending action to be completed and then try unfreezing the file again.

Deleting Frozen Files

If one or more frozen files need to be permanently removed from the IDA service, it is possible to delete both individual frozen files as well as all frozen files within the scope of a particular folder.

The delete action cannot be terminated before it is complete. Depending on the size and number of frozen files being deleted, the background operations may take several hours. The status of the pending delete action can be viewed in the pending actions listing as described below.

WARNING: Deleted files will be permanently and irretrievably removed from the IDA service, and will no longer be accessible to other services, making all external references to them invalid. If the files are linked to datasets defined in Qvain, deleting deprecates these datasets permanently. A warning dialog will be presented if any datasets would be deprecated by the delete action. If frozen files are deleted, all replicated copies will be removed. The delete action, once initiated, cannot be undone!

To delete a frozen file or all frozen files within the scope of a folder, navigate in the files view to the location in the frozen area of the file or folder and click on the snowflake icon to the right of the file or folder name, which will open the action tab on the right side of the view. Then, in the action tab, click on the button “Delete”.

In the confirmation dialog presented, if you are absolutely sure you wish to delete the selected file(s), click “Yes”.

Once the delete action has been successfully initiated, you will be presented with a confirmation, and by clicking on the pending action link shown in the dialog, you may open the action summary (described in greater detail below).

Note that the delete action will be rejected if any pathname of any of the files to be unfrozen conflicts with a file belonging to a pending action. This is to permit all essential background processing to finish before like named files are deleted, to avoid potential confusion or disruption in other services which may be utilizing frozen file metadata in some manner.

If you wish to delete a file which shares the same relative pathname as a file in a pending action, you must first wait for the pending action to be completed and then try deleting the file again.

Temporary Share Links

You may create download-only temporary share links to either individual files or entire folders, which will be valid for a period of at most 90 days, and may optionally be assigned a required password to access, as well as other options described below.

Temporary links may be created for files or folders in either the staging or frozen area.

This is useful for providing temporary access to data, either during final review of data in the staging area, prior to freezing, or data in the frozen area prior to inclusion in and publication of a dataset.

WARNING: Temporary share links should not be published nor used in citations or as public download links! They are intended for limited, short-term, unofficial use only.

To create a temporary share link for a file or folder, navigate in the files view to the location of the file or folder and click on the share icon to the right of the file or folder name, which will open the sharing tab on the right side of the view. Then, in the sharing tab, click the plus “+” icon to the right of the label “Share link” to add a new link.

A pull-down menu will appear, showing the expiration date of the newly created link in 90 days, and offering the option of specifying a shorter expiration period if desired by clicking on the predefined 90 day expiration date and selecting an earlier date. To create the link, click “Create share”, else click “Cancel”.

Once the link is created, it will be automatically copied to the browser clipboard, so that you may provide the link to whomever you wish to allow temporary download-only access to the data via that link.

Additionally, you can further configure the link options via the ellipsis menu to the right of the link.

To optionally require a password to access the data, click “Password protect” and enter the desired password in the field provided, and press “Return”.

To change the expiration date, select a new date by clicking on the date field.

To optionally provide a note which will be visible to external users using the link, click “Note to recipient” and enter the desired note in the field provided, and press “Return”.

The link URL may be later copied again to the clipboard by clicking on the copy icon to the left of the ellipsis menu of the link.

It is not possible to make changes to or delete shared data through a temporary sharing link.

When using the temporary download link, note that the multi-select actions in the pull down at the top of the files view have no knowledge of which folders and files are read-only, so the move/copy/delete options are available, but they will fail if the user does not have sufficient privileges, which is the case for all temporarily shared data. The folder/file specific actions in the ellipsis menu to the right of the folder/file name show the actual actions available for each folder/file (which for temporary shared data is download only).

To disable an existing temporary share link, open the link options pull-down menu by clicking the ellipsis menu of the link, and select “Unshare”. The link will be immediately deactivated and discarded.

To create an additional share link, e.g. with different link options, select “Add another link” from the ellipsis menu of the link.

WARNING: There is no confirmation before disabling a temporary share link. Disabling a temporary share link cannot be undone and the previous share link cannot be retrieved. If you still desire the file or folder to have a temporary share link, a new share link will need to be created by checking the option, and provided to all persons who should use the link.

Note that temporary share links are specific to each IDA user, and cannot be seen nor modified by other project members. It is the responsibility of each project member to inform other project members about, and coordinate the creation and management of, temporary share links, and to observe any organization and/or project-specific policies concerning such links.

Favorites

Files or folders which are frequently in use can be marked as “favorites” so that they can be accessed more easily and directly than browsing to their location in the files view.

Favorite files and folders are easily accessed by selecting the “Favorites” view from the left hand navigation of the files view.

To mark a file or folder as “favorite”, click the star icon to the left of the file or folder name. The star will become highlighted and the file or folder will be included in the “Favorites” view.

To remove a file or folder from the “Favorites” view, click the highlighted star icon to the left of the file or folder name.

Action Listings

The IDA service performs numerous background operations to maximize the responsiveness of the service and the efficiency of processes, and ultimately minimize the amount of time users need to spend interacting with the service.

The IDA service provides easy access to listings of actions, ordered by most recently initiated to least recently initiated, and organized by status as described below.

To view all available action listings, click on “ACTIONS” in the top navigation bar.

The default action listing will be displayed, showing all pending actions.

To view the details of a particular action shown in a particular listing, click on the action identifier link. The time at which each individual background operation was completed will be shown in the top right of the summary. If the action is a failed action, the cause for the failure will be clearly indicated in red. The files associated with the action will be listed in the lower part of the summary.

Pending Actions

Once initiated, an action is pending until all background operations are completed, or some unrecoverable error is encountered. As individual operations are completed, the status of the pending action is updated accordingly, and can be monitored by regularly updating the summary view for the action.

To view all pending actions, open the actions listing section by clicking on “ACTIONS” in the top navigation bar as described above, or if already in the action listings section and another action status listing is shown, click “Pending Actions” in the left hand navigation.

Completed Actions

After all background operations have been completed successfully, an action is complete.

To view all completed actions, open the actions listing section by clicking on “ACTIONS” in the top navigation bar as described above, if not already in the action listings section, and click “Completed Actions” in the left hand navigation.

Failed Actions

If any background operation fails to complete successfully, the action is flagged as a failed action, and the cause of the failure will be clearly indicated in the action summary.

To view all failed actions, open the actions listing section by clicking on “ACTIONS” in the top navigation bar as described above, if not already in the action listings section, and click “Failed Actions” in the left hand navigation.

Retrying Failed Actions

Failed actions may be retried, such that the sequence of background operations will be resumed from the point that the previous operation failed. This assumes that the underlying cause of the failure has been resolved, possibly with the assistance of the IDA support staff.

When retrying a failed action, the original failed action will be marked as “cleared” and a new action will be created, based on the failed action, and initiated. The original action summary will then show a link to this new retry action. In this manner, an action may be retried any number of times, with the multiple actions linked to each attempt accordingly (as the retried action may either succeed and be marked completed, or fail for the same or some other reason).

To retry a failed action, click “Retry” to the right of the action in the failed action listing, or within the action summary of the failed action.

Cleared Actions

Cleared actions are failed actions which have either been retried or simply abandoned. If the action was retried, a link to the retry action will be present in the action summary.

To view all cleared actions, open the actions listing section by clicking on “ACTIONS” in the top navigation bar as described above, if not already in the action listings section, and click “Cleared Actions” in the left hand navigation.

Notification icons

Notification icons will appear in the top navigation bar, to the right of the Actions menu option, when any of the following conditions apply:

	There are one or more pending actions
	There are one or more failed actions
	A project is suspended (locked) and in read-only mode

Note that if a user belongs to multiple projects, the relevant icons will appear if the condition exists for any of the projects to which the user belongs.

Clicking on a notification icon will open the appropriate actions listing view, via which more information will be available, such as the project with which each pending or failed action is associated, providing links to the action details, or which project is suspended.

Collision Avoidance for File Operations

All users belonging to a given project have the same rights, and may interact with, add, and remove project data concurrently. To help avoid one user unintentionally interfering with another user’s activity, the IDA service employs a number of checks and restrictions to ensure that multiple concurrent users’ activities do not collide in undesirable ways.

Under normal circumstances, encountering any of these potential collisions should be very rare; however, guarding against such potential collisions helps to maintain data integrity in both the staging and frozen areas.

Command Line Tools

Tools for interacting with data via the command line are available for download from a public GitHub repository: https://github.com/CSCfi/ida2-command-line-tools. They are compatible with most UNIX-like platforms (Linux, OS-X, etc),

Uploading, downloading, moving (renaming), and deleting of files and folders to/from a project’s staging area are supported. Downloading frozen data and retrieving information about frozen files is also supported.

Freezing files and unfreezing or deleting frozen files, however, require use of the IDA web UI as described above, and these actions cannot be performed from the command line.

To get you started, see video tutorials for Installation and Configuration, Uploading and Validation of Data and for Working with Frozen Data.

App passwords

The IDA Command Line Tools must be provided with valid IDA account credentials in order to access the project space. The username will be your CSC account username, as managed in the CSC Customer Portal https://my.csc.fi/. The password will be an application password that you create in the IDA service. Instructions for creating a app password can be found in IDA Command Line Tool GitHub repository.

Using IDA with other CSC services

If you’re using IDA with CSC supercomputers or Allas, you can find instructions also from CSC Docs:

• Using IDA with Puhti (CSC Docs)
• Accessing IDA via Puhti’s and Mahti’s graphical web interface (CSC Docs)
• Copying data between Allas and IDA via Puhti (CSC Docs)

If you have any questions about particular features, or any other issue relating to the service, please contact IDA support at servicedesk (a) csc.fi