simple-cloud-kit

Simple Cloud Kit

About

Exclusive Information Technology Service is pleased to offer a tool to be used by AWS Cloud Engineers within enterprise environments that will provide opinionated methodologies for how AWS Multi-Account Landing Zones should be implemented to maintain CIS (Critical Infocom Service) regulatory compliance and Best Practices.

We call this kit, the Simple Cloud Kit. And will refer to it as SCK.

We will refer to the core services within the SCK as core-automation or in many cases simply core.

Current Stage: development - incubation

The Model

The model for the enterprise is based on the concept of seperation of duties. Each task is conducted by a person having a persona’s such that each of the teams responsible for governance of infrastructure services may operate the Landing Zones safely and in collaboration with other teams.

Description

Persona's are **Hats**. When you work on infastructure, you will need switch *hats* to perform duties with the Simple Cloud Kit. By switching hats, it will begin to understand the structure of the service and command set.

Persona’s

Billing / Finance Team This team is responsible for the Accounting, Billing, and payments for AWS within the enterprise.

AWS Account Access: AWS Organization Account

Typical subsystems that this team will access are:

Identity Team This team is accountable for all Users (i.e. human users) that need to access internal infrastrcture.

AWS Account Access: AWS Identity and Access Management Account

The services are typically Active Directory services or Kerberose and includes all process of PIAM (Priviledged Identity/Access Management). In many cases, this includes the SSL Certificate generation for both server certificates and user certificates and manages the Certificate Authority for the enterprise.

Typical subsystems that this team will access are:

Domain Team Manages all names or domains. You will need to understand both your internal domain and external domain name requirements.

AWS Account Access: AWS Network Account

External Domains This is your domain name as recognized by your external customers. External customers might access your services at “www.mycompany.com”. The Domain team determines and provides names for use within landing zone external access points.

Internal Domains The domain team issues internal domain names for your corporate LAN or Intranet. It might be a subdomain for your primary domain. Options are limitless. But we can imagine a subdomain such as “myservice.internal.mycompany.com” or “myservice.internal.mycompany.net”. Again, this is the descretion of the Domain Team.

Cloud Engineers must interact with the Domain Team to establish the paramters for automation and infrastructure deployment. We must deploy infrastructure into the proper domain. The access to the *AWS Network Account** is to register internal domain names into the landing zone.

In your organization, the Domain Team may be responsible for administration of Windows Domain Controllers. If so, then they needs to understand Route53 as these are Domain Controllers extinging their domain services. As domains are registered in the Domain Controllers, they must be regisgtered in the AWS Landing Zones Route53.

Domains and Subdomains registred in the AWS Landing Zones is managed completely by the SCK internal services.

Typical Subsystems access by this team are:

Firewall Team The firewall team will be responsible for addressing ingress control from the Internet to the landing zones as well as internal corporate systems that need connectivity to Cloud services.

AWS Account Access: DMZ Account and Network Accounts

This team will interact with services such as.

Their purpose is to grant or revoke access to services that have been deployed. This team is typically capable of configuring the services. They are Users and the automation must enable this team to perform the required business function.

Monitoring / Audit / Logging Team This team is responsible for setting up and operating the logging services for all applications and infastructure. They do not provide the metrics. Instead, they provide the services that teams use to record their metrics.

AWS Account Access: Audit and Logging Account

In many cases, you may have a monitoring Command Center (COM) (SOC) that needs to use these services as well to monitor alarms.

This team will provide the logging and compliance requirements for each classification of services.

Typical subsystems that this team will use are:

As the Audit/Logging team requirs “Encryption At Rest”, KMS keys are typically managed by this team. This includes all KMS keys geneerated in automation serivces. However, some organizations might wish this to be managed byt eh Sucrity Team or the Identity team and the SCK can be configured to accomodate.

Secuirty and Compliance Team The security and compliance team needs the ability to priovide SIAM solutions across all infrastructure deployed in the enterprise. This is how CIS and security compliance is monitored and verified.

AWS Account Access: AWS Secuirty Accounts

The enterprise may operate a Security Operations Center or SOC that are tasked with early and immediate response to security threats and vulnerabilities. This is not monitoring, this is security compliance. Intrusion events are managed here as well as IDS and IPS system administration deployed across all accounts in the Landing Zones.

Typical Systems that this team leverages are:

Network Infrastructure Team This team might be considered the parent team of the Cloud Operations Team. The network infastructure team is the foundation of Landing Zones and provides the rule of engagement. Cloud Operations cannot be implemented in the enterprise with Networks.

AWS Account Access: AWS Network Accounts

The Networks team provides governance of all Landing Zone deployments and network expansion in the enterprise. Cloud Operations along with the Simple Cloud Kit automation frameworks provides network expansion a.k.a Landing Zones allocating CIDR space to new infrastructure and determining how network traffic traversed the Intranet enterprise network.

Typical services this team will leverage:

Application Support Team The application support team has several responsibilitie to ensure appication infrastructure and app;ication support services is deployed.

AWS Account Access: AWS Automation Account and Shared Services Accounts

The Application services teams are Delivery Support. They manage several subsystems to support application deployment. This can include:

This team may be more than one sub-team. It depends on the size of the organization and the enterprise infrastructure support model. This team may have 1 or more Shared Services Zones depending on its security requirements

DevSecOps Team The DevSecOps team manages all of the tools necessary to govern software development with the entrprise.

AWS Account Access: AWS Automation Account or DevSecOps Account (only one, not both)

The DevSecOps team provides the tools and support for systems such as:

This team may or may not be part of the Cloud Operations team as it depends on the organization requirements. There is a “circular dependency” between CloudOps and DevOps as both need each other to bootstrap the environments. The SCK implementation model atempts to address this.

AWS Automation Team This is the Cloud Engineering and Operations Team. The entire mandate for this team is to provide automation frameworks, operational guidelines, and expertiese to the above named infrastructure teams and to design the deployment templates for Cloud infastructure deployment.

AWS Account Access: AWS Automation Account and ALL AWS Accounts

The automation team installs, configures, and operates the Simple Cloud Kit. If there are customizations and new configuration parameters to define, the Cloud Engineering team makes those modifications and deploys new versions of the SCK.

Fundamentally, the Cloud Entineering Team is both Accountable and Responsible to deploy ALL infrstructure in the Cloud for ALL of the above teams. In order to do this efficiently, it creates robots (the SCK) to perform these tasks in an automated way:

Typical Systems Needed

AWS Accounts for Landing Zones

There are 9 accouts to bootstrap in the AWS Multi-Account Landing Zones architecture which is what the Simple Cloud Kit is designed for.

These accounts support the Persona’s listed above and are:

AWS Organization Account

Noteable Services Manually created by the Organization

AWS Audit/Logging Account

Audit/Log Services Automatically by Control Tower

AWS Security/Compliance Account

Security Services Automatically by Control Tower

AWS Identity Account

Identity Services

AWS DMZ Account

Ingress/Egress Services

AWS Networking Account

Networking Services

AWS Shared Services Account

Application Services

AWS DevSecOps Account

Automation Services

AWS Automation Account

Cloud Engineering Services

What’s Next

Simple Cloud Kit

STATE: INCUBATION

As this project is currently in incubation, components of the arhitecture will be presented periodically and staged. The Simple Cloud Kit is composed of the following components and will be in subdirectories within this repsitory.

You will be able to find this project at https://github.com/eitssg/simple-cloud-kit

The basic structure of the repsitory will be:

Repository: simple-cloud-kit Version: 0.0.0-alpha

Other Projects

simple-cloud-kit-ui

This is a UI/Dashboard to deploy on AWS to allow GUI popuplation and generation of templates for applications and assistance provisioning landing zones an new infrastructure deployment pipeliens

Installation

# python -m venv .venv
# source .venv/bin/activate
# pip install simple-cloud-kit
# core --help
  Simple Cloud Kit (c) 2024 Exclusive Information Technology Service
  Core 0.0.1-pre01934jf
  ...help text (coming soon!!!)

You can make core available in your PATH if you wish. Here is how I would do it (Installation program comming soon)

In the “simple-cloud-kit” folder, create a new subfolder called “bin”, copy “core.exe” into it, and then add that folder to your path:

cd simple-cloud-kit
mkdir bin
cp .venv/bin/core.exe bin/core.exe
CWD=$(cwd)
echo "export PATH=$PATH:/$CWD/bin" >> ~/.bashrc

Do something like above or similar. Then you can run “core” from anywhere. Even your own custom projects.

Setup / Contributing

Setup

step 1

Clone the repositry:

git clone https://github.com/eitssg/simple-cloud-kit.git
cd simple-cloud-kit
python -m venv .venv

There are 14 git submodules in this repo. Sync all the submodules and pull all the subprojects

step 2

In IntelliJ or VSCode, select this python as the interpreter.

install poetry

source .venv/bin/activate
pip install poetry poetry-dynamic-versioning

Next, evaluate the build tool scripts for windows (.ps1) powershell, or mac/linux (.sh) bash (not zsh or sh…bash)

In vsCode or Intellij, add each of the 14 submodules to the workspace. (adding in the ‘path’)

Switch all project TOML files to “develop” mode by setting project dependeces “develop=true” in the 14 TOML files.

python ./prebuild.py

If you wish to switch back to “publish” production mode, open versions.json and set the develop attribute to false and re-run the prebuild.py script

run “build-all.sh” to install and build all submodules and install all dependencies

source ./build-all.sh

Talk to me via Github or email (jbarwick@eits.com.sg)

Simple Cloud Kit Dashboard

STATE: INCUBATION

In addition to the Simple-Cloud-Kit, the team is devloping a User Interface dashboad as a separate project that will leverage the sck-mod-core API as a REACT application.

We will call this the Simple Cloud kit UI or SCK Dashboard

You will be able to find this project at https://github.com/eitssg/simple-cloud-kit-ui

The basic structure of the repsitory will be:

Repository: simple-cloud-kit-ui Version: 0.0.0-alpha

As refactoring and testing of each component is developed, it will be pushed to this repository for your consumption.

Licensing

The Simple Cloud Kit (SCK) will be presented under the GPL-3.0 Licensing framework.

Why GPL? Why not MIT?

GPL-3.0 was selected as the license of choice to ensure we as a community begin sharing code to create the Holy Grail of cloud deployment engines. We need a huge amount of feedback and requirements gathering. And, procuding this under MIT would mean “the code is yours”. We want this code to belong to The Community. Thus, GPL-3.0 is the choice.

More about the GPL-3.0 license can be found here: https://www.gnu.org/licenses/gpl-3.0.en.html

We do hope you will be able to contribute to this project.

Authors

This premise and format of this project has been developed of the years by Sourced Group which is now part of the Amdocs team. I used to work for Sourced Group.

The foundation of the concepts came from a version of Core-Automation developed by Sourced Group in 2019. Their original code is unlicensed or MIT license (Freeware).

I assure you, this will be a COMPLETE REWRITE. Although Core-Autmation and its methodologies was the inspiration, apart from terminology, there will be only a few simiarities in templates and structure as the idea is to confirm to Best Practices wich I learned by working with them.

As templates are Jinja2 and CloudFormation, there may be some copying of templates. But the underlying main engines and Python is from scratch.

If there are any questions regarding this source code, please reach out to the primary maintainer:

James Barwick email: jbarwick@eits.com.sg

Contributing

Please contribute. Please reach out to me if you would like to create a development branch that we can ultimately merge into the base.

All modules can be found on GitHub with the following links:

PyPi Repositories can be found here:

Documentation

The idea on this project is to push documentation to GitHub type Wiki or perhaps I can get a free Atlassian OSS Conluence site. For all the details and goodies of implementing, using, and updating the core frameworks, see the online documentation:

TBD

… (c) Copyright 2024. Exclusive Information Technology Service, c/o James Barwick

This site is open source. Improve this page.