Neolithic Populous Metropolis

    my-ci-stack

    0.1.3 • Public • Published

    OpenSearch Continuous Integration

    OpenSearch Continuous Integration is an open source CI system for OpenSearch and its plugins.

    Getting Started

    • Requires NPM to be installed
    • Install project dependencies using npm install from this project directory
    • Configure aws credentials
    • Deploy stacks with npm run cdk deploy

    Deployment

    Dev Deployment

    1. Setup your local machine to credentials to deploy to the AWS Account

    2. Deploy the bootstrap stack by running following command that sets up required resources to create the stacks. More info

      npm run cdk bootstrap -- -c useSsl=false -c runWithOidc=false

    3. Deploy the ci-config-stack using the with one of the following (takes ~1 minute to deploy) -

      npm run cdk deploy CI-Config-Dev -- -c useSsl=false -c runWithOidc=false

    4. Optional Configure the elements of the config stack for SSL configuration

    5. Optional Configure the elements setting up oidc via federate

    6. Deploy the ci-stack, takes ~10 minutes to deploy (parameter values depend on step 2 and step 3)

      npm run cdk deploy CI-Dev -- -c useSsl=false -c runWithOidc=false

    7. Log onto the AWS Console of the account, navigate to cloud watch, open log groups, looking for JenkinsMainNode/var/log/jenkins/jenkins.log

    8. Search the logs for Jenkins initial setup is required. An admin user has been created and a password generated. After that entry the password for the jenkins instance will be in the cloudwatch logs.

    9. Go to the CI-Dev.JenkinsExternalLoadBalancerDns url returned by CDK output to access the jenkins host.

    10. If you want to destroy the stack make sure you delete the agent nodes manually (via jenkins UI or AWS console) so that shared resources (like vpc, security groups, etc) can be deleted.

    Executing Optional Tasks

    SSL Configuration

    1. Locate the secret manager arns in the ci-config-stack outputs
    2. Update the secret value (see docs) for the certContentsSecret with the certificate contents
    $aws secretsmanager put-secret-value \
    --secret-id MyTestDatabaseSecret_or_ARN \
    --secret-string file://mycreds.json_or_value
    
    1. Update the secret value (see docs) for the privateKeySecret with the certificate private key
    2. Upload the certificate to IAM see docs
    3. Update the secret value for the certificateArnSecret with the certificate arn generated by IAM
    4. Run with parameter using one of the following (refer this for value of runWithOidc)
      1. npm run cdk deploy CI-Dev -- -c useSsl=true -c runWithOidc=true or,
      2. cdk deploy CI-Dev -c useSsl=true -c runWithOidc=true
    5. Continue with next steps

    Setup OpenId Connect (OIDC) via Federate

    1. Locate the secret manager arns in the ci-config-stack outputs
    2. Update the secret value (see docs) for the OIDCClientIdSecret with the credentials as json as follows:
      1. JSON format
       {
           "clientId": "example_id",
           "clientPassword": "example_password",
           "wellKnownOpenIDConfigurationUrl": "https://www.example.com",
           "tokenServerUrl": "https://example.com/token",
           "authorizationServerUrl": "https://example.com/authorize",
           "userInfoServerUrl": "https://example.com/userinfo"
       }
      
      1. Command Eg: see docs
       $aws secretsmanager put-secret-value \
       --secret-id MyTestDatabaseSecret_or_ARN \
       --secret-string file://mycreds.json_or_value
      
    3. Run with parameter with one of the following (refer this for value of useSsL) -
      1. npm run cdk deploy CI-Dev -- -c runWithOidc=false -c useSsl=true or,
      2. cdk deploy CI-Dev -c runWithOidc=false -c useSsl=true
    4. Continue with next steps

    Troubleshooting

    Main Node

    Useful links

    Useful commands

    • npm run build compile typescript to js, run lint, run tests
    • npm run watch watch for changes and compile
    • npm run cdk deploy deploy this stack to your default AWS account/region
    • npm run cdk diff compare deployed stack with current state
    • npm run cdk synth emits the synthesized CloudFormation template

    Architecture Overview

    Plantuml diagram, see ./diagrams/opensearch-ci-overview.puml for source

    Built using AWS Cloud Development Kit the configuration of the CI systems will be available for replication in your own accounts. The Jenkins instance will be hardened and publically visible, connected to GitHub to make build notifications easy for everyone to see.

    Contributing

    See developer guide and how to contribute to this project.

    Getting Help

    If you find a bug, or have a feature request, please don't hesitate to open an issue in this repository.

    For more information, see project website and documentation. If you need help and are unsure where to open an issue, try forums.

    Code of Conduct

    This project has adopted the Amazon Open Source Code of Conduct. For more information see the Code of Conduct FAQ, or contact opensource-codeofconduct@amazon.com with any additional questions or comments.

    Security

    If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our vulnerability reporting page. Please do not create a public GitHub issue.

    License

    This project is licensed under the Apache v2.0 License.

    Copyright

    Copyright OpenSearch Contributors. See NOTICE for details.

    Keywords

    none

    Install

    npm i my-ci-stack

    DownloadsWeekly Downloads

    1

    Version

    0.1.3

    License

    none

    Unpacked Size

    240 kB

    Total Files

    65

    Last publish

    Collaborators

    • sayaligaikawad