ACCESS Model Config Files: Inventory & Parser Support

Introduction: The Quest for Seamless Experiment Generation

Hey guys, let's dive into a crucial aspect of our work: ensuring the ACCESS experiment generator is super robust and versatile. This means it needs to play nice with all the different model types we use, like ACCESS-OM2, ACCESS-OM3, and ESM1.5/1.6. To make this happen, we need a solid understanding of all the configuration files involved. This article is all about creating a definitive list of these files, figuring out which ones need special attention in terms of parsing and updating, and making sure our experiment generator can handle them all.

We're not just talking about reading the files, but also being able to tweak them and write the changes back without messing up the format or structure. This is where the parser and updater functionalities come into play. The parser is the brain that reads and understands the file, while the updater is the muscle that modifies and rewrites it. Without these, our experiment generator would be pretty limited, unable to adapt to various configurations and setups. So, let's explore the files that demand these capabilities to ensure our experiment generation is as flexible and powerful as possible.

This is an ongoing project, and we're constantly refining our understanding of these files and their roles in the models. The more complete our inventory, the better equipped we are to build a truly versatile experiment generator. This effort involves identifying files, assessing their parsing and updating requirements, and implementing the necessary tools to support them. This includes the creation of parsers to interpret file formats and the development of updaters to modify these files. By doing so, we can ensure our experiment generator effectively manages the diverse configuration needs of the ACCESS models.

Detailed Configuration File Inventory and Analysis

To clarify the configuration files, here's a detailed table. It pinpoints which files need parsers and updaters for each model type. We'll break down each file type, specifying which models use them and the current status of parser and updater support. We'll also note any special characteristics or considerations for each file to get a comprehensive view of the configuration landscape. This table provides a clear view of each file type and its status across different model types, highlighting where our work is focused. The goal is to offer a precise, up-to-date view of our progress and what's coming up.

📌 Required for the given model – Not applicable ✅ Implemented ❌ Not implemented 🛠️ Work in progress in a related PR

YAML Files: The Configuration Cornerstone

YAML files are super important in the ACCESS ecosystem, acting as the backbone for configuring many different aspects of our models. They're used extensively in Payu configurations, making them a critical part of our experiment setup. YAML's human-readable format makes them easy to understand and modify, which is essential for both parsers and updaters. We’ve got good news here: both the parser and updater support for YAML files are fully implemented (✅). This means our experiment generator can read, interpret, and modify these files without any issues. This is a big win, as YAML files are so fundamental to how our models work. The ability to seamlessly handle YAML configurations ensures we can adapt and run experiments smoothly. This support ensures flexibility in how we set up and run experiments, allowing us to easily change parameters and configurations.

_in and .nml Files: Fortran Namelists Demystified

Next up, we have _in and .nml files, which are essentially Fortran namelists. These files are used across all our model types (OM2, OM3, and ESM1.5/1.6), making them incredibly important. Good news: We have implemented both parser and updater functionality for these files. The experiment generator can read, modify, and write back to these files while keeping their structure intact. This is essential because namelists are used to control a wide array of model behaviors and settings. By supporting these files, we ensure that users can easily customize and experiment with different configurations. The ability to automatically modify these files saves time and reduces the chances of errors.

MOM_input, nuopc.runseq, nuopc.runconfig: MOM6 and NUOPC Specifics

These three file types are more specialized, tailored for specific components of the ACCESS models. MOM_input is specifically for MOM6, while nuopc.runseq and nuopc.runconfig are crucial for the NUOPC coupler. The parser and updater are both implemented for these. It means the experiment generator can handle the unique configurations required by these specific components. This specialized support ensures our experiments can work with the latest model versions and setups.

XML Files: Data Forcing Configuration

XML files are used for configuring data forcing. Currently, neither the parser nor updater is implemented for these files (❌). We need to enhance this part to improve the model's data handling capabilities. Implementing parser and updater support for these files is a future goal.

Forcing.json: Special OM2 Forcing File

This file is a special case, mainly used for OM2. Currently, the parser and updater are both a work in progress (🛠️). This means we're actively working on enabling the experiment generator to handle the specific forcing configurations used in OM2. Supporting forcing.json will enhance our experiment setup capabilities for OM2 models.

Field_table and Data_table: Ocean Model Internal Declarations

Finally, we have field_table and data_table, which are critical for ocean model operations. Currently, neither parser nor updater support exists for these files (❌). These files declare variables and look up forcing data, which is essential for running ocean models. Adding support for them will boost the capabilities of the experiment generator. These files are crucial for defining how the model handles variables and external data.

The Road Ahead: Implementation and Collaboration

Parser Development and Integration

The parsers that handle these configuration files are currently located in src/experiment_generator/tmp_parser. The plan is to swap these out with more robust versions from ACCESS-NRI/access-parsers once they are ready. This transition will provide a significant upgrade, improving the efficiency and reliability of our experiment generation. The goal is to have a unified and more powerful parsing system. This integrated system will support a wide array of configuration files, streamlining the entire process.

Collaboration and Future Improvements

We need to expand the table to include more ESM1.5/1.6 config files. This is where you guys come in! Please contribute by adding the missing files and updating the table using the same merged-emoji format. Your input is super valuable! We welcome contributions from everyone who works with these models. Additionally, please create GitHub issues in this repo to share use cases. This helps us prioritize improvements and ensure we're addressing the most important needs. The more we collaborate, the more efficient and user-friendly our experiment generator becomes. Sharing your experiences and ideas will help us improve the tool and make it more effective for everyone.

By continuously refining and expanding the experiment generator, we can ensure that it remains a cutting-edge tool for climate modeling research. This is crucial for the success of our projects.