We were tasked with preparing a presentation that describes constraints that I am experiencing in my business. The presentation must also describe how the application of methods/tools/techniques learned during the course will enable me to better manage this situation.
This final week was very nerve-wrecking and very interesting for me. It was nerve-wrecking because I have a severe speech impediment (a stutter) that may unnecessarily cause me to take longer than the other students. So, I decided to record my presentation to ease this process. It was also very interesting because I work in a very complex organization that is part of a very dynamic and international Science and Technology project that is experiencing but also addressing complex challenges. It was exciting for me to outline these issues and also providing my proposed solutions to these challenges.
Background and Overview
Inter-University Institute for Data Intensive Astronomy (IDIA) was born out of the partnership of University of the Western Cape (UWC), University of Cape Town (UCT) and North-West University (NWU). These three institutions built this partnership on the building blocks of what is known was the African Research Cloud (ARC). The ARC was created in 2015 when UCT and the NWU worked together to build a research cloud. The ARC is a cloud-based infrastructure hosted by UCT & NWU which lead to the Astronomy Proof of Concept, which was led by researchers from IDIA. The Astronomy POC involved the development of a data intensive calibration and imaging pipeline for radio telescopes, with an emphasis on MeerKAT. The pilot of this initiative is ARCADE.
The ARC tested different models of data management, storage and transfer with the aim of supporting the data processing steps required to transform raw MeerKAT data into scientific data projects. The success of the ARC ultimately led to the large-scale deployment of the pipeline on the IDIA facility, a facility built in anticipation of the deluge of MeerKAT data from different Large Survey Projects.
In September 2015 a formal partnership was agreed upon between UCT, NWU and UWC which lead to the official establishment of IDIA. In February 2016 the University of Pretoria (UP) joined the partnership. It is in that same year that the grant for Ilifu was successfully proposed and accepted by the Data Intensive Research Institute of South Africa (DIRISA) by six partner institutions through its National Integrated Cyberinfrastructure System (NICIS) to build a data-centric computing system that will provide computing power and data storage for projects in the strategic fields of astronomy and bioinformatics. It was at this point that IDIA resources and personnel focused on both Astronomy and Bioinformatics users.
You can learn more about IDIA in the video below:
Constraints
IDIA is engaged in the development of data processing pipelines for the MeerKAT, the precusor the Square Kilometre Array (SKA). The SKA is positioned to be the worlds largest telescope that will generate astronomical datasets, excuse the pun. These datasets will be the largest ever produced by a single telescope and could go up to 100 000 000 Tera Bytes by 2030. These datasets will be processed by a SKA Super Computer at the SKA Observatory in the United Kingdom. This SKA Super Computer will be augmented by two other supercomputers known as the Science Data Processor (SDP) based in Cape Town, South Africa and Perth, Australia. The Centre for High Performance Computing and the South African Radio Astronomy will be actively involved in bringing the SDP to life.
The SDP will process science data from the SKA telescope into astronomical images which are the science data products of interest to Astronomers. The SDP will be linked to the SKA Super Computer at the SKA Observatory where the SKA Observatory will generate 600 PB of calibrated science data products each year.
According Huynh (2019)
"the data rates and volumes from central signal processor are so high we can not store the raw data from it. It is cheaper to re-observe than store the raw data indefinitely. The science data processor becomes a schedulable resource for the telescope for observation planning."
According to Huynh (2019) constraints experienced in the Astronomy community as it relates to the SKA are :
- Complexity;
- Scalability, Extensibility, Lifetime;
- Cost; and
- Power
In 2016 the SKA Board identified a gap between construction cost and support cost. These support costs include advanced analytics products, user support for smaller projects and teams, archiving of Science results for future research. This led to recommendation that Science Regional Centres (SRCs) be established as essential elements of the science delivery. My presentation will focus on the costs constraint as it was concluded the SRCs were outside of the scope of the SKA1 project (e.g.precursors such as MeerKAT). The costs of the SDP are constrained by capital cost (e.g. SKA1 is at R13 Trillion in 2016 for the project, and the SDP is constrained to 10% of that capital cost).
Huynh (2019) provides a cost of estimate of SRC activities in Europe, Africa, China, Australia, India and Canada. In this costing overview IDIA is observed to have received the least amount of money than any other organization in the project. For instance, IDIA receives +R13 Million as compared to CIRADA in Canada that receives +R130 Million (10 times more). Given that SRCs are not funded by the SKA project but by individual nations, costs are a constraint to the project.
Application of Tools and Methodologies
To address this constraint we can use tools as the Business model and Customer Empathy Map. The current Business Model could be reviewed with the objective of developing a futuristic view of the business that will respond to the challenges. This new business model will define the business architectures and technologies relevant to South African SRC at a national and international level. This business model will enable the matching of computing infrastructure to support user workflows, and an accurate estimate of processing and storage requirements and costs.
This new business model will be a business case for funding by the South African government, similar to the Australian SKA Regional Centre, and allows for a better view of institute. The new business model should focus on the SRC essential functions as defined by the SKA Board (Square Kilometre Array Organization, 2016)
New View of the Business
IDIA's business model, displayed in the image below, will be re-configured as follows:
- Customer Segments
- Astronomers
- General Public
- Customer Relationships
- Value Proposition
- Community-Wide Access (Astronomers)
- Reprocessing of Big Data
- Key Activities
- SKA Science Data Products
- Data Re-processing & Science Analysis
- Long-Term Science Archival
- User Support
- Computational Resource Management
- Education and Outreach
- Key Partners
- These will largely stay the same
- Will introduce the SKA Observatory as a new partner
- Cost Structures
- Shared Services
- Cost Optimization
- Revenue Streams
- New revenues from the Appropriation of National Government revenues
Proposed Business Model for IDIA |
This new business model will fit well into the Tiered Science Delivery Model outlined by Ratcliffe (2019) and it is displayed below. Further to this, this new business model will align itself to the core functions of an SRC. According to Square Kilometre Array Organization (2016) "SKA Regional Centres (SRCs) to provide the essential functions that are not within the scope of the SKA project: specifically, computational capacity for re-processing and science analysis, provision of an SKA Science Archive".
Tiered Science Delivery (Ratcliffe, 2019) |
User Story
A user story will enable us to understand how users will use a regional centre. This is critical as no SKA user should care where their data products are, and all SKA users should be able to access their data products, irrespective of whether their country or region hosts a regional centre. Below we will visualize the Customer Empathy Map.
References
Huynh, M., 2019. CERN Event: The Square Kilometre Array Computing From SDP to SRC. [Online]
Available at: https://indico.cern.ch/event/773049/contributions/3581362/attachments/1937710/3211780/SKA_SDP_SRC_CHEP_Huynh_upload.pdf
[Accessed 11 September 2020].
Ratcliffe, S., 2019. SKA Regional Centre: The South African Perspective. [Online]
Available at: https://indico.skatelescope.org/event/559/contributions/6230/attachments/5295/7347/ratcliffe_srcsc_kickoff_may_2019.pdf
[Accessed 11 September 2020].
South African Radio Astronomy Observatory, 2019. Media Releases: SKA Consortium Completes Design of Science Data Processor. [Online]
Available at: https://www.sarao.ac.za/media-releases/ska-consortium-completes-design-of-science-data-processor/
[Accessed 11 September 2020].
Square Kilometre Array Organisation, 2020. Software and Computing. [Online]
Available at: https://www.skatelescope.org/software-and-computing/
[Accessed 11 September 2020].
Square Kilometre Array Organization, 2016. SKA Regional Centre Steering Committee (SRCSC) Terms of Reference, Manchester: Square Kilometre Array Organization.