Eugene de Beste

Senior Cloud Engineer

“The right man in the wrong place can make all the difference in the world.”

- G-man, Half-Life 2.

Current Role

Senior Cloud and HPC Technologist II

In my current role, I am based at the Centre for High-Performance Computing’s (CHPC) Advanced Computer Engineering (ACE) Lab, with a primary focus on HPC education, cloud computing systems, and distributed storage. The CHPC operates under the National Integrated Cyber Infrastructure System (NICIS), which is part of the prestigious Council for Scientific and Industrial Research (CSIR) of South Africa.

Responsi­bilities

In my capacity as a Senior Cloud Technologist, I am entrusted with a diverse range of responsibilities, including but not limited to:

• • Cloud Computing Research and Development: Spearheading research on OpenStack, managing the ACE Lab’s OpenStack cloud, and leading the OpenStack Research Cloud (ORC) initiative for innovative solutions.
• • CSIR Public Cloud Offering Policies: Shaping and implementing CSIR’s public cloud offering policies with seamless technical support.
• • Third-Party Collaboration: Collaborating with academic and SMME partners to develop private cloud capabilities and upskilling.
• • Automation Engineering: Optimizing efficiency through infrastructure-as-code practices and advising CSIR on new paradigms.
• • GitOps Implementation and Infrastructure Modernization: Introducing GitOps for infrastructure management and enhancing deployment schemes.
• • Distributed Storage Research: Managing and optimizing the ACE CEPH cluster for potential tier 1 filesystem use in HPC.
• • International Student Cluster Competition (SCC) Initiative: Co-planning and supporting the SCC initiative for educating South African students in Linux and HPC.
• • Infrastructure Procurement and Management: Efficiently managing infrastructure procurement, vendor communication, and documentation.
• • Software Innovations: Implementing advanced monitoring systems, contributing to the software stack, and enhancing inventory management.
• • Support and Training: Providing technical support and training for staff on cloud computing, networking, software containers and more.
• • Supporting the South African Development Community (SADC): Contributing to HPC equipment donations, education, and training across Africa.
• • Network Configuration and Debugging: Managing network configurations and facilitating new equipment installations.
• • Containerization and Continuous Delivery: Driving the adoption of containerization and continuous delivery practices for software development.
• • Graduate Staff Training: Training graduate staff in various cloud computing and HPC concepts.
• • Technical Expertise Support: Offering insights on networking, cloud operations, and related matters to various business units.

Special Projects

NB: Details of projects listed here are reduced in work experience section below to remove redundancies.
CHPC = Centre for High Performance Computing (South Africa) | UCT = University of Cape Town | SANBI = South African National Bioinformatics Institute | HISP = Health Information Systems Program | NICD = National Institute for Communicable Diseases | ICS = Internet Communication Services | HPC = High Performance Computing | UWC = University of the Western Cape

African Pathogen Archive

CHPC In (2023 - ongoing), I facilitated the signing of a Memorandum of Understanding (MoU) between the CHPC and SANBI, in which I am assuming the role of Infrastructure Automation Lead for the African Pathogen Archive (APA) project. My key responsibilities include:

• • Porting microservice application stack to a local development environment.
• • Designing and developed automation for APA services.
• • Rapidly deploying the APA stack with Kubernetes on OpenStack using Terraform and Kubespray.
• • Implementing GitOps with FluxCD v2 for streamlined management.
• • Organizing project’s git structure for efficient collaboration.
• • Conceptualizing and working on deployable black-box OpenStack + APA stack units for federation across Africa.

HISP OpenStack Deploy­ment

CHPC In (2022), I established a significant collaboration between HISP and the CHPC for consulting on an OpenStack private cloud deployment. My responsibilities encompassed a range of duties, including:

• • Optimizing hardware infrastructure for OpenStack deployment.
• • Conceptualizing and designing the network layout and configured switches for OpenStack control plane.
• • Configuring network switching equipment, including VLAN access and MLAG on Mellanox switches.
• • Deploying the MAAS provisioning server with high-availability.
• • Successfully deploying OpenStack platform using Kolla and Kolla-ansible.
• • Providing comprehensive training and documentation to the team.

Student Cluster Compe­tition

CHPC In (2020), I played a crucial role in the CHPC’s adaptation of the Student Cluster Competition to an online format during the challenging times of the coronavirus pandemic. My duties included:

• • Creating high-quality educational videos on essential topics. YouTube playlist as example.
• • Preparing and organizing content on the learning management system.
• • Providing expert assistance and support to students.

Ilifu Project

UCT/SANBI During (2018 - 2020), I actively participated in the Ilifu project, a data-intensive research cloud designed for academic research, specifically in the fields of astronomy and bioinformatics. This collaborative venture involved multiple universities and the Department of Science and Technology in South Africa. My responsibilities encompassed various aspects of planning, prototyping, and deploying an OpenStack-based private cloud with a multi-petabyte CEPH cluster for storage. Key duties included:

• • Collaborating with the UCT team to meticulously plan the deployment of the private cloud.
• • Creating and implementing a prototype OpenStack instance on testing hardware.
• • Planning and prototyping an OpenStack-based private cloud with CEPH storage.
• • Successfully developing and deploying the production OpenStack private cloud.
• • Exploring and implementing OpenStack Manila to enable the use of the CEPH for end-users of the cloud.
• • Providing consulting services and second-tier technology support for the CEPH implementation.
• • Contributing to the overall success of the project aimed at advancing data-intensive research in the academic community.
• • For more details, visit the project website: http://www.ilifu.ac.za/.

HISP OpenStack Deploy­ment Training

SANBI From (2019 - 2020), I collaborated with HISP in facilitating the deployment of an OpenStack cluster. My responsibilities included:

• • Introducing software containers and conducting comprehensive training sessions for HISP users to ensure their proficiency in this area.
• • Planning the hardware and network layout for the HISP OpenStack cluster, engaging in discussions to understand specific needs and expectations for the cloud environment.
• • Strategizing the deployment of the OpenStack cloud and providing extensive training to the HISP IT staff, enabling them to effectively manage and utilize the OpenStack infrastructure.
• • Collaborating with the HISP IT team to execute multiple deployment scenarios in a simulated lab environment, offering hands-on training for OpenStack deployment.

NICD HPC Deploy­ment

SANBI Between (2019 - 2020), I was actively involved in the deployment of an HPC software stack at the NICD. My key responsibilities included:

• • Conducting a thorough evaluation and understanding of the NICD’s hardware configuration to ensure compatibility with the HPC software stack.
• • Deploying the HPC software stack at NICD with Slurm and Singularity containers.
• • Facilitating deployment of Galaxy platform on the HPC software stack.
• • Providing comprehensive training to NICD IT staff.

UWC HPC Deploy­ment

SANBI During (2019 - 2020), I collaborated with UWC eResearch to facilitate the deployment of an HPC cluster, aiming to provide unified research-focused resources to researchers and students across various departments. My responsibilities included:

• • Collaborating with UWC eResearch and ISC to deploy an HPC cluster.
• • Planning of hardware and software stack and repurposing of donated hardware.
• • Implementing and utilizing MAAS for bare-metal server deployment.
• • Developing Ansible playbooks and using them to deploy a Slurm-based cluster.
• • Providing comprehensive support for software and hardware stacks.

Professional Work Experience

Job Roles

2020-ongoing (Current) Senior Cloud and HPC Technologist II — Advanced Computer Engineering Lab, Centre for High Performance Computing, Council for Scientific and Industrial Research

• Refer to the Responsibilities section in Current Role above.

---

2018-2020 External Consultant — ILIFU Project, University of Cape Town

• • Planned, developed, and implemented a tailored private cloud deployment for astronomy and bioinformatics researchers.
• • Provided expert consulting and ongoing support for the private cloud infrastructure.
• Refer to Ilifu Project under the Special Projects section for more detal.

2017-2020 Systems Developer — South African National Bioinformatics Institute, University of the Western Cape

• • Trained student teams for the Centre for High Performance Computing’s Student Cluster Competition, some of which placed first locally and second internationally. (Source.)
• • Migrated in-house developed VM management software to OpenStack Cloud.
• • Trained and assisted on the implementation of an OpenStack based cloud to the Health Information Systems Program (HISP) organisation.
• • Maintained, optimized and enhanced CEPH storage, both from a system and user perspective.
• • Implemented the software stack for an HPC cluster and Galaxy server at the National Institute for Communicable Diseases (NICD).
• • Deployed an HPC cluster for UWC eResearch in collaboration with UWC Internet Communication Services (ICS).
• • Containerized (Docker) system and administrative services for IT staff efficiency.
• • Containerized (Singularity) many bioinformatics scientific user applications.
• • Migrated to automated infrastructure deployments using MAAS, PXE, Ansible, and Terraform.
• • Implemented pfSense firewall to migrate legacy system which involved reworking SANBI networking configuration.
• • Deployed HPC clusters in cloud and on bare-metal (with focus on the SLURM scheduler.).
• • Wrote many Ansible roles and playbooks for automated deployment of various internal systems.
• • Migrated authentication LDAP system to FreeIPA.
• • Implemented monitoring automation.
• • Managed IT infrastructure, provided consulting and support to users, and handled procurement.
• • Established change management control and knowledge repositories in the IT team.
• • Network debugging and tracing (VLANs, Openvswitch, port mappings).
• • Implementation of a helpdesk (Freshdesk) ticketing system for IT staff to track user issues and generate reports.

2015-2016 Systems Administrator Intern — VOSS Solutions

• • Upgraded production service software and managed VMWare, AD, DNS, and WSUS.
• • Provided user management on Cisco ASA and Linux devices.
• • Assisted with quotes, purchase orders, and offered desktop support for employees.

Volunteering Roles

2017-2018 Volunteer Staff — BSides Cape Town

• Assisted with event preparation and operations.

2017-2018 Volunteer Staff — DevOpsDays Cape Town

• Assisted with event preparation and operations.

2018 Tutor — JEDI Workshop, Madagascar (Source.)

• Trained students in Jupyter notebooks, Bash, Linux navigation, Docker, Singularity, and developing reproducible data processing pipelines in astronomy context.

2017 Tutor — Software Carpentry, Observatory

• Assisted in tutoring researchers in Bash, Python, and Git at the Software Carpentry meeting during PyCon 2017.

2016 Hackathon Participant — H3ABioNet Cloud computing hackathon, University of Pretoria (Source.)

• Co-developed the Nextflow implementation of the H3Agwas pipeline for H3ABioNet, Dockerizing tools and using Nextflow. GitHub Repository here.

2014 Tutor — Software Carpentry, University of Cape Town (Source.)

• Assisted in tutoring researchers in Bash, Python, and Git at the Software Carpentry meeting at the University of Cape Town.

Education

Tertiary Education

2016-2019 M.Sc. Bioinformatics — South African National Bioinformatics Institute, University of the Western Cape

• Awarded Cum Laude.

2015 B.Sc. Hons Information Technology — University of Cape Town

• • Mini-thesis: de Beste, E., Stewart, G. and Gain, J. (2015). PyTeacher: Determining the Usefulness of Visual Learning for Computer Science.
• • Built a League of Legends statistics analysis platform for class on Visual Thinking and Visualisation. (Source.)
• • Awarded B.Sc. Hons in Information Technology Cum Laude.

2012-2014 B.Sc. Computer Science — University of the Western Cape

• Awarded B.Sc. Computer Science Cum Laude.

Secondary Education

2007-2011 National Senior Certificate — Fairbairn College

Academic Outputs

Journal Publications

2018 Ahmed, A.E., Mpangase, P.T., Panji, S., Baichoo, S., Souilmi, Y., Fadlelmola, F.M., Alghali, M., Aron, S., Bendou, H., de Beste, E. and Mbiyavanga, M. (2018). “Organizing and running bioinformatics hackathons within Africa: The H3ABioNet cloud computing experience”. AAS open research, 1. (Source.)

Thesis (M.Sc)

2019 de Beste, E. (2019). “Enabling the processing of bioinformatics workflows where data is located through the use of cloud and container technologies”. (Source.)

Posters

2017 de Beste, E., van Heusden, P., Christoffels, A., Bagula, A. (2017). “Moving Workfows to Data”. ISCB Africa ASBCB Conference on Bioinformatics. (Source.)

2019 de Beste, E., Christoffels, A., van Heusden, P. (2019). “Leveraging Open Source Technologies to Rapidly Deploy Infrastructure with Minimum Human Capacity”. Centre for High Performance Computing Annual Conference. (Source.)

Confer­ences

2013

• Centre for High Performance Computing Annual Conference — Cape Town, South Africa
• Participated in the local CHPC Student Cluster Competition, achieved first place.

2014

• International Supercomputing Conference — Leipzig, Germany
• Participated in the international Student Cluster Challenge, achieved first overall prize.
• Software Carpentry UCT — Cape Town, South Africa
• Tutored researchers in scientific computing with BASH and Python.

2016

• Centre for High Performance Computing Annual Conference — East London, South Africa
• Mentored UWC student team in the local Student Cluster Competition, achieved first place locally and went on to achieve second place internationally.

2017

• Software Carpentry PyCon — Cape Town, South Africa
• Teaching assistant and tutor for scientific computing.
• BSides Cape Town — Cape Town, South Africa
• Volunteered as staff for organizing the event.
• ISCB Africa ASBCB Conference on Bioinformatics — Entebbe, Uganda
• Presented a poster on “Moving Workfows to Data” (refer to poster section).
• Global Emerging Pathogens Treatment Consortium (GET) workshop on Data Management Design and LIMS Usage — Freetown, Sierra Leone
• Presented work on MSc. thesis, cloud, and container technologies for bioinformatics.
• DevOpsDays Cape Town — Cape Town, South Africa
• Volunteered as staff for organizing the event.

2018

• JEDI Workship and Conference — Madagascar
• Led workshop on reproducible science using software containers and pipeline/workflow tools.
• DevOpsDays Cape Town — Cape Town, South Africa
• Volunteered as staff for organizing the event.
• BSides Cape Town — Cape Town, South Africa
• Volunteered as staff for organizing the event.

2019

• DevOpsDays Cape Town — Cape Town, South Africa
• Volunteered as staff for organizing the event.
• Supercomputing Conference — Denver, Colorado, United States of America
• Awarded opportunity to attend through the STEM-TREK program in collaboration with CHPC.
• Centre for High Performance Computing Annual Conference — Johannesburg, South Africa
• Presented a poster on “Leveraging Open Source Technologies to Rapidly Deploy Infrastructure with Minimum Human Capacity” (refer to poster section).

2022

• International Supercomputing Conference — Frankfurt, Germany
• Attended various talks and workshops related to cloud and HPC synergy.

Achieve­ments

2018 BSides Cape Town - Rite of Passage — Las Vegas, United States of America

• Selected to attend DefCon cyber-security conference in Las Vegas as part of the “Rite of Passage” program for volunteering at BSides Cape Town event. I contributed to event management and gained access to elite hacker talks and demonstrations at DefCon.

2014 First Place Overall Prize for the ISC’14 Student Cluster Challenge — International Supercomputing Conference, Leipzig, Germany

• Led a team to victory at the ISC’14 Student Cluster Challenge in Germany, excelling in hardware and software cluster design and high-performance computing tasks.

Top 3 Academic Achiever in Third Year — University of the Western Cape (Source.)

• Ranked third among Computer Science bachelor’s degree students in the third year.

2013 First Place for the CHPC Student Cluster Competition 2013 — Centre for High Performance Computing, Council for Scientific and Industrial Research

• Secured first place in the CHPC Student Cluster Competition, demonstrating excellence in systems administration and cluster competition.

IBM Master the Mainframe Competition — University of the Western Cape

• Successfully completed the first phase of the IBM Master the Mainframe Competition, showcasing proficiency in operating mainframe computing environments.

2012 Golden Key Society — University of the Western Cape

• Invited to join the Golden Key International Honour Society, recognizing high academic achievers and providing valuable benefits.

Top 5 Academic Achiever in First Year — University of the Western Cape (Source.)

• Ranked among the top 5 academic achievers in the first year of the Computer Science bachelor’s degree.

Occu­pation

Cape Town, South Africa

The Mother City