ccd-docker

KO | EN

gitlite — search

#javascript #hacktoberfest #react #ai #typescript #deep-learning #awesome #cli #awesome-list #pytorch #security #mcp #ios #openai

(★ 11)

No description available.

ccd-docker Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# CCD Docker :whale:
- [Prerequisites](#prerequisites)
- [Quick start](#quick-start)
- [Configuring CCD](#configuring-ccd)
- [Running branches](#running-branches)
- [Enabling additional projects](#enabling-additional-projects)
- [Under the hood](#under-the-hood-speedboat)
- [Local development](#local-development)
- [Troubleshooting](#troubleshooting)
- [Variables](#variables)
- [Remarks](#remarks)

## Additional Documentation
- [Azure Setup Guide](/docs/Azure.md)
- [Containers explained](/docs/Containers.md)
- [IDAM Alternatives](/docs/IdamAlt.md)
- [Running on Apple Silicon (ARM64)](/docs/AppleSilicon.md)
- [Windows Subsystem for Linux configuration](/docs/WSL.md)
- [Migrate PostgreSQL database to latest](/docs/PostgresUpgrade.md)
- [License](/LICENSE.md)

## Prerequisites

- [JDK 17](https://openjdk.java.net/projects/jdk/17/)
- [Docker](https://www.docker.com)

> [!Note]
  > *Once docker is installed, increase the memory and CPU allocations (Docker -> Preferences -> Advanced) to at least the following minimum values for successful execution of ccd applications altogether:*
  >
  > | *Resources* | Memory   | CPU<sup>s</sup>  |
  > | :---: | :------: | :---: |
  > | Minimum | 7+ GB   | 4+    |
  > | Full IDAM + XUI | 12+ GB | 6+ |
  > | Reccomended | 14+ GB | 7+ |
  >
  > For Windows Subsystem for Linux (WSL & WSL2) users [see this page](./docs/WSL.md) for how to do this.

- [Azure CLI](/docs/Azure.md) - minimum version 2.0.57
- [jq Json Processor](https://ghcr.io/jqlang/jq)
- [psql / libpq](/docs/psql.md)
- Mac users, set your default shell to bash 
  ```bash
  chsh -s /bin/bash
  ```
  If this doesn't work, then try changing it in the Terminal app's settings under
  ```
  General > Shells open with: > Command (complete path) > /bin/bash
  ```

## Quick start

*The following documentation assumes that the current directory is `ccd-docker`.*

### 1. Checkout `ccd-docker` and `ccd-definition-store-api` projects:

```bash
git clone git@github.com:hmcts/ccd-docker.git
```

### 2. For Extra Quick setup on Mac/Linux

For customised setups please ignore this step and [continue setup at step 3](#3-authenticate-azure-and-login-to-the-azure-container-registry)

Run the following command to setup using default services/configuration and simulated idam 
```bash
./ccd default
```
OR Run the following command to setup using default services/configuration and the full idam stack 
```bash
./ccd default idam
```
OR Run the following command to setup using default services/configuration and simulated idam plus xui frontend
```bash
./ccd default xui
```
After command is run if no extra changes need to be made then you should be good 
to go and can skip the rest.
If any additional scripts need runnig then make sure the environoment variables are set, 
as defined in [Export environment variables](#b-export-environment-variables)

### 3. Authenticate Azure and login to the Azure Container registry:
Make sure Azure CLI is installed, if not got to [Azure Setup Guide.](/docs/Azure.md) Then login to azure and the ccd container repositories using the commands below

```bash
az login
./ccd login
```
### 4. Migrate existing Postgres DB
> [!IMPORTANT]
> ⚠️ **THIS STEP IS ONLY REQUIRED IF YOU NEED TO MIGRATE POSTGRES VERSION** ⚠️

If ccd-docker has been previously setup, images/volumes and the database may point to an earlier Postgres version. Prior to pulling images run the below commands to **delete** existing images and volumes.

```
./ccd compose down
```

See [Guide on migrating postgres](/docs/PostgresUpgrade.md) for more details

### 5. Pull latest Docker images
   
```bash
./ccd compose pull
```

### 6. Set up network and configuration

> [!Note]
> Required only on the first run. Once executed, it doesn't need to be executed again

#### A. Create docker network
  
```bash
  ./ccd init
  ```
Ignore if we get error message ccd-network already exists while running above command
  
#### B. Export environment variables

CDM apps require a set of environment variables which can be set up by executing the following script.
  
##### Windows
  ```bash
  ./bin/set-environment-variables.sh
  ```
  
##### Linux / Mac 
  ```bash
  source ./bin/set-environment-variables.sh
  ```
  > [!CAUTION]
  > Some users of zsh 'Oh My Zsh' experienced issues.
  > Try switching to bash by : `chsh -s /bin/bash`

**Important environment notes:**
  - `OIDC_ISSUER` must be derived from a real access token for the target environment. Do not guess it from the public OIDC discovery URL.
  - `OIDC_ALLOWED_ISSUERS` is an optional comma-separated additive list of exact JWT `iss` values. Leave it unset unless a service is confirmed to receive valid tokens from more than one issuer.
  - `CCD_CALLBACK_ALLOWED_HOSTS` is the comma-separated allow-list of HTTPS callback target hosts CCD services may call.
  - `CCD_CALLBACK_ALLOWED_HTTP_HOSTS` is the comma-separated allow-list of HTTP callback target hosts CCD services may call.
  - `CCD_CALLBACK_ALLOW_PRIVATE_HOSTS` controls whether callbacks to private or local hostnames are allowed for local development.

**How to derive `OIDC_ISSUER`:**
  - Do not guess the issuer from the public discovery URL alone.
  - Decode only the JWT payload from a real access token for the target environment and inspect the `iss` claim.
  - If additional issuers are required, derive each `OIDC_ALLOWED_ISSUERS` value from a real accepted token too.
  - Do not store or document full bearer tokens. Record only the derived issuer value.

Example:
  ```bash
  TOKEN='eyJ...'
  PAYLOAD=$(printf '%s' "$TOKEN" | cut -d '.' -f2)
  python3 - <<'PY' "$PAYLOAD"
  import base64, json, sys
  payload = sys.argv[1]
  payload += '=' * (-len(payload) % 4)
  print(json.loads(base64.urlsafe_b64decode(payload))["iss"])
  PY
  ```
  - JWTs are `header.payload.signature`.
  - The second segment is base64url-encoded JSON.
  - This decodes the payload only. It does not verify the signature.
  The following services in `ccd-docker` compose set both `IDAM_OIDC_URL` and `OIDC_ISSUER`:

| Service |
  | --- |
  | `ccd-data-store-api` |
  | `ccd-definition-store-api` |
  | `cpo-case-payment-orders-api` |
  | `ts-translation-service` |
  | `ccd-case-document-am-api` |

The following services also pass through `OIDC_ALLOWED_ISSUERS` when it is exported in the shell:

| Service |
  | --- |
  | `ccd-data-store-api` |
  | `ccd-definition-store-api` |
  | `ccd-case-document-am-api` |
  | `cpo-case-payment-orders-api` |
  | `ts-translation-service` |

`VERIFY_OIDC_ISSUER=true` is not set in this repo's compose YAML. Use it only in service repos that include a live issuer verifier, where it enables a pre-check that fetches a real test token and fails fast if its `iss` claim does not exactly match `OIDC_ISSUER` or an explicitly configured `OIDC_ALLOWED_ISSUERS` value.
  
  To persist the environment variables in Linux/Mac run the following script
  to add the script into your ~/.bash_profile.
  ``` bash
  ./bin/add-to-bash-profile.sh
  exec bash -l
  ```

#### C. Enabling ElasticSearch if needed
  Additionally, export these environment variables to enable ElasticSearch

```bash
  export ELASTIC_SEARCH_ENABLED=true
  export ES_ENABLED_DOCKER=true
  ```

### 7. Creating and starting the containers:

```bash
./ccd compose up -d
```

---

## Configuring CCD

Once the containers are runnning, CCD's admin web can be accessed at [https://localhost:3100](https://localhost:3100).
If using xui-frontend it ca be accessed at [https://localhost:3455](https://localhost:3455).

However, some more steps are required to correctly configure CCD before it can be used:

> [!NOTE]
> All scripts require the following environment variables to be set
> - IDAM_ADMIN_USER
> - IDAM_ADMIN_PASSWORD

If they are not working then check you have run `source ./bin/set-environment-variables.sh` correctly.
If still not working then try setting them directly with the following commands.
```bash
export IDAM_ADMIN_USER=idamOwner@hmcts.net
export IDAM_ADMIN_PASSWORD=Ref0rmIsFun
```
And check they match the corresponding values from the confluence page at https://tools.hmcts.net/confluence/x/eQP3P

### CCD Quick Start

At this point most users can run the following 3 scripts

```bash
./bin/add-users.sh && 
./bin/add-ccd-roles.sh && 
./bin/add-role-assignments.sh
```

Create CCD users and roles

a. Clone `ccd-definition-store-api` if not already checked out `git clone git@github.com:hmcts/ccd-definition-store-api.git`
and navigate to the `ccd-definition-store-api`.

b. Run smoke tests to set up user and roles.

```bash
export TEST_URL=http://localhost:4451

./gradlew clean smoke

```

> [!Note] 
> incase of any errors relating to Service Auth when running the smoke test, ensure the following environment variable is set as below:

export IDAM_S2S_URL=http://service-auth-provider-api:8080

Alternatively remove this from the environment by issuing 'unset IDAM_S2S_URL', backend.yaml will default to use 'http://service-auth-provider-api:8080'

./ccd compose up -d

Will need to be re-issued to apply the above changes.
]

The smoke tests creates a file `/aat/befta_recent_executions_info.json`, delete this file after running the tests.

---

Move on to the [Ready for take-off](###Ready-for-take-off) section.

---
A more in depth explanation of the scripts is detailed below

### 1. Add users

A script is provided that sets up some initial users and roles for running functional tests. Execute the following:

```bash
# FOR IDAM FULL STACK
./bin/add-users.sh
```

This script will add the users with associated roles as defined in

```bash
bin/users.json
```

This script runs the checks below, for each user defined in the `users.json`

```bash
check roles
if roles are the same
    do nothing
else
    delete user
    create user with same id
```

Therefore to
 * add a new user - add a new entry to the `users.json`
 * modify an existing user - modify `users.json` to add/remove a role

Alternatively, add a user to SIDAM by using the script

```bash
# FOR IDAM-SIM
./bin/utils/idam-simulator-create-user.sh ROLE EMAIL_ADDRESS LAST_NAME FIRST_NAME
# FOR IDAM FULL STACK
./bin/idam-create-caseworker.sh ROLE EMAIL_ADDRESS LAST_NAME FIRST_NAME
```

> [!NOTE]
> LAST_NAME if omitted defaults to `TesterLastName`
>
> FIRST_NAME if omitted defaults to `TesterFirstname`
>
> Password for each user created by the script defaults to `Pa55word11`

You may verify the service has been added by logging in to the SIDAM Web Admin with the URL and
logic credentials here:

https://tools.hmcts.net/confluence/x/eQP3P

Navigate to

`Home > Manage Users`

and search for users by email address.

### 2. Add CCD roles

Execute the following script to add roles to CCD:

```bash
# FOR IDAM FULL STACK
./bin/add-ccd-roles.sh
```

The script parses `bin/ccd-roles.json` and loops through a list of roles and their security classifications, passing the values to the `ccd-add-role.sh` script.

By default most FTA (Feature test automation) packs load their own roles into CCD via the definition store each time
the feature tests are run

To add a further role to CCD (by importing it into the definition store), run the following script

```bash
# FOR IDAM FULL STACK
./bin/ccd-add-role.sh
```

supplying the following parameters

```bash
- role: Name of the role. Must be an existing IDAM role.
- classification: Classification granted to the role; one of `PUBLIC`,
        `PRIVATE` or `RESTRICTED`. Default to `PUBLIC`.
```

For example, to add the `caseworker` role (that must exist in SIDAM) to CCD, use

```bash
# FOR IDAM FULL STACK
./bin/ccd-add-role.sh caseworker PUBLIC
```

### 3. Add role assignments

A script is provided that sets up some initial role assignments for users. Execute the following:

```bash
# FOR IDAM FULL STACK
./bin/add-role-assignments.sh
```

This script will add role assignments for the associated users as defined in

```bash
# FOR IDAM FULL STACK
bin/am-role-assignments.json
```

This script runs the checks below, for each user defined in the `am-role-assignments.json`

```bash
if overrideAll flag is true
    delete any existing role assignments for the user
else 
    do nothing

then 
    create all user's assignments
```

Note that currently the role assignments are created in the role assignment database directly due to restrictions in the rules engine of the Role Assignment Service APIs.

The `am-role-assignments.json` can be modified to add further role assignments to users.

### 6. Import case definition

#### Note:

CCD Data Store FTA tests will automatically import the CCD case definitions using the `befta-fw` test framework.

Case definitions can be imported using CCD's dedicated UI

Case definitions can also be imported manually via the command line, using the following script

```bash
./bin/ccd-import-definition.sh <path_to_definition>
```

Parameters:
- `path_to_definition`: Path to `.xlsx` file containing the case definition.

> [!Note] 
> For CCD to work, the definition must contain the caseworker's email address.

If the import fails with an error of the form:

```
Validation errors occurred importing the spreadsheet.

- Invalid IdamRole 'caseworker-cmc-loa1' in AuthorisationCaseField tab, case type 'MoneyClaimCase', case field 'submitterId', crud 'CRUD'
```

Then the indicated role, here `caseworker-cmc-loa1`, must be added to CCD (See [2. Create Idam roles](2-.-Create-Idam-roles)).
### Ready for take-off 🛫

Back to [http://localhost:3451](http://localhost:3451), you can now log in with any of the email addresses defined when adding users in [3. Create Users](#3.-Create-users).
All user passwords default to : `Pa55word11`.

If you see only a grey screen after entering your user credentials in the login page, you may need to set profile settings in ccd_user_profile database by adding a single line for the user in the below tables:

1- user_profile

2- user_profile_jurisdiction

---

## Running branches

By default, all CCD containers are running with the `latest` tag, built from the `master` branch.

### Switch to a branch

Using the `set` command, branches can be changed per project. It's possible to switch to remote branches but also to local branches

To switch to a remote branch the command is:

```bash
./ccd set <project> <remote_branch>
```

* `<project>` the service name as declared in the compose file, e.g. ccd-data-store-api, ccd-test-stubs-service
* `<remote_branch>` must be an existing **remote** branch for the selected project.

To switch to a local branch the command is:

```bash
./ccd set <project> <local_branch> <file://local_repository_path>
```
* `<project>` the service name as declared in the compose file, e.g. ccd-data-store-api, ccd-test-stubs-service
* `<local_branch>` must be an existing **local** branch for the selected project.
* `<file://local_repository_path>` path to the root of the local project repository

__Note__: when working with local branches, to be able to run any new set of local changes those must first be committed and the `switch to a local branch` and `Apply` procedure repeated.

Branches for a project can be listed using:

```bash
./ccd branches <project>
```

### Apply

When switching to a branch, a Docker image is built locally and the Docker compose configuration is updated.

However, to make that configuration effective, the Docker containers must be updated using:

```bash
./ccd compose up -d
```

### Revert to `master`

When a project has been switched to a branch, it can be reverted to `master` in 2 ways:

```bash
./ccd set <project> master
```

```bash
./ccd unset <project> [<projects...>]
```

The only difference is that `unset` allows for multiple projects to be reset to `master`.

In both cases, like with the `set` command, for the reset to be effective it requires the containers to be updated:

```bash
./ccd compose up -d
```

### Current branches

To know which branches are currently used, the `status` command can be used:

```bash
./ccd status
```

The 2nd part of the output indicates the current branches.
The output can either be of the form:

> No overrides, all using master

when no branches are used; or:

> Current overrides:
> ccd-admin-web branch:RDM-2414 hash:ced648d

when branches are in use.

> [!TIP]
> In addition to the `status` command, the current status is also displayed for every `compose` commands.

---

## Enabling additional projects

By default, `ccd-docker` runs the most commonly used backend and frontend projects required by CCD:

* Back-end:
  * **sidam-api**: Strategic identity and access control
  * **service-auth-provider-api**: Service-to-service security layer
  * **ccd-user-profile-api**: Users/jurisdictions association and usage preferences
  * **ccd-definition-store-api**: CCD's dynamic case definition repository
  * **ccd-data-store-api**: CCD's cases repository
  * **ccd-test-stubs-service**: CCD's testing support for stubbing http calls (service callbacks etc)
  * **am-role-assignment-service**: Users' role assignments for access management
  * **cft-hearing-service**: Hearing Service API
* Front-end:
  * **idam-web-public**: SIDAM's login UI
  * **ccd-api-gateway**: Proxy with SIDAM and S2S integration

Optional compose files will allow other projects to be enabled on demand using the `enable` and `disable` commands.

### Note on Running Optional Projects

If this is the first time running these optional projects from ccd docker you will need to do `./ccd compose pull` after you have enable the optional project

Also if a certain database has not been created you might need to create a new ccd shared database image. To do this you need to do this:

* Run `/ccd compose down -v`
* Then find the id of any ccd shared database using `docker images | grep ccd-shared-database`
* Using the ids from previous command use  `docker image rm <id>`
* Then run `./ccd compose up -d`

### Optional Project Commands

* To enable **document-management-store-app**
  * `./ccd enable backend frontend dm-store`
  * run docker compose `./ccd compose up -d`
  * create Blob Store in Azurite `./bin/dm-store/document-management-store-create-blob-store-container.sh`

* To enable **ExUI** rather then the CCD UI
  * `./ccd enable xui-frontend`
  * export XUI_LAUNCH_DARKLY_CLIENT_ID to value mentioned in xui web app preview template yaml file. i.e. 645baeea2787d812993d9d70
  * run docker compose `./ccd compose up -d`
  * access ExUI at `http://localhost:3455`

* To enable **ElasticSearch**
  > [!Warning] 
  > We recommend at lest 16GB of memory for Docker when enabling elasticsearch
  * `./ccd enable elasticsearch` (assuming `backend` is already enabled, otherwise enable it)
  * export ES_ENABLED_DOCKER=true
  * verify that Data Store is able to connect to elasticsearch: `curl localhost:4452/health`

* To enable **Logstash**
  * `./ccd enable logstash` (assuming `elasticsearch` is already enabled, otherwise enable it)
  * Note that the config for Logstash is contained within the [logstash directory](logstash)

* To enable **ccd-message-publisher**
  * NOTE: By default the CCD Message Publisher will use an embedded ActiveMQ instance. See [ccd-message-publisher](https://github.com/hmcts/ccd-message-publisher) for more information.
  * `./ccd enable backend message-publisher`
  * Run docker compose `./ccd compose up -d`
  * Verify that ccd-message-publisher is up and running by `curl localhost:4456/health`

* To enable **ccd-case-disposer**
  * `./ccd enable backend case-disposer`
  * Run docker compose `./ccd compose up -d`

* To enable **ccd-next-hearing-date-updater**
  * `./ccd enable backend ccd-next-hearing-date-updater`
  * Run docker compose `./ccd compose up -d`

* To enable **ccd-case-document-am-api**
  * `./ccd enable backend frontend dm-store case-document-am`
  * run docker compose `./ccd compose up -d`
  * verify that ccd-case-document-am-api is up and running by `curl localhost:4455/health`

* To enable **ts-translation-service**
  * `./ccd enable backend ts-translation-service`
  * run docker compose `./ccd compose up -d`
  * verify that ts-translation-service is up and running by `curl localhost:4650/health`

* To enable **cft-hearing-service**
  * `./ccd enable backend hearings`
  * run docker compose `./ccd compose up -d`
  * verify that cft-hearing-service is up and running by `curl localhost:4651/health`
  * this will include the inbound and outbound adapters

* To enable **hmc-operational-reports-runner**
  * `./ccd enable backend operational`
  * run docker compose `./ccd compose up -d`
  * verify that hmc-operational-reports-runner is up and running by `curl localhost:4651/health`

---

## Under the hood :speedboat:

### Set

#### Non-`master` branches

When switching to a branch with the `set` command, the following actions take place:

1. The given branch is cloned in the temporary `.workspace` folder
2. If required, the project is built
3. A docker image is built
4. The Docker image is tagged as `hmcts/<project>:<branch>-<git hash>`
5. An entry is added to file `.tags.env` exporting an environment variable `<PROJECT>_TAG` with a value `<branch>-<git hash>` matching the Docker image tag

The `.tags.env` file is sourced whenever the `ccd compose` command is used and allows to override the Docker images version used in the Docker compose files.

Hence, to make that change effective, the containers must be updated using `./ccd compose up`.

#### `master` branch

When switching a project to `master` branch, the branch override is removed using the `unset` command detailed below.

### Unset

Given a list of 1 or more projects, for each project:

1. If `.tags.env` contains an entry for the project, the entry is removed

Similarly to when branches are set, for a change to `.tags.env` to be applied, the containers must be updated using `./ccd compose up`.

### Status

Retrieve from `.tags.env` the branches and compose files currently enabled and display them.

### Compose

```bash
./ccd compose [<docker compose command> [options]]
```

The compose command acts as a wrapper around `docker compose` and accept all commands and options supported by it.

> [!TIP]
> For the complete documentation of Docker Compose CLI, see [Compose command-line reference](https://docs.docker.com/compose/reference/).*

Here are some useful commands:

#### Up

```bash
./ccd compose up [-d]
```

This command:
1. Create missing containers
2. Recreate outdated containers (= apply configuration changes)
3. Start all enabled containers

The `-d` (detached) option start the containers in the background.

#### Down

```bash
./ccd compose down [-v] [project]
```

This stops and destroys all composed containers.

If provided, the `-v` option will also clean the volumes.

Destroyed containers cannot be restarted. New containers will need to be built using the `up` command.

#### Ps

```bash
./ccd compose ps [<project>]
```

Gives the current state of all or specified composed projects.

#### Logs

```bash
./ccd compose logs [-f] [<project>]
```

Displays the logs for all or specified composed projects.

The `-f` (follow) option allows to follow the tail of the logs.

#### Start/stop

```bash
./ccd compose start [<project>]
./ccd compose stop [<project>]
```

Start or stop all or specified composed containers. Stopped containers can be restarted with the `start` command.

> [!warning] 
> Re-starting a project with stop/start does **not** apply configuration changes. Instead, the `up` command should be used to that end.

#### Pull

```bash
./ccd compose pull [project]
```

Fetch the latest version of an image from its source. For the new version to be used, the associated container must be re-created using the `up` command.

### Configuration

#### OAuth 2

OAuth 2 clients must be explicitly declared in service `sidam-api` with their ID and secret.

A client is defined as an environment variable complying to the pattern:

```yml
environment:
  IDAM_API_OAUTH2_CLIENT_CLIENT_SECRETS_<CLIENT_ID>: <CLIENT_SECRET>
```

The `CLIENT_SECRET` must then also be provided to the container used by the client service.

> [!TIP]
> To prevent duplication, the client secret should be defined in the `.env` file and then used in the compose files using string interpolation `"${<VARIABLE_NAME>}"`.

#### Service-to-Service

Micro-services names and secret keys must be registered as part of `service-auth-provider-api` configuration by adding environment variables like:

```yml
environment:
  MICROSERVICE_KEYS_<SERVICE_NAME>: <SERVICE_SECRET>
```

The `SERVICE_SECRET` must then also be provided to the container running the micro-service.

> [!TIP]
> To prevent duplication, the client secret should be defined in the `.env` file and then used in the compose files using string interpolation `"${<VARIABLE_NAME>}"`.

#### Address lookup

To use UK address lookup feature an API key for https://postcodeinfo.service.justice.gov.uk is required. When API key is available it needs to be set on host side under `ADDRESS_LOOKUP_TOKEN` variable name.

---

## Local development

The provided Docker compose files can be used to get up and running for local development.

However, while working, it is more convenient to run a project directly on the localhost rather than having to rebuild a docker image and a container.
This means mixing a locally-run project, the one being worked on, with projects running in Docker containers.

Given their unique configuration and dependencies, the way to achieve this varies slightly from one project to the other.

Here's the overall approach:

### 1. Update containers to point to local project

As is, the containers are configured to use one another.
Thus, the first step to replace a container by a locally running instance is to update all references to this container in the compose files.

For instances, to use a local data store, references in `ccd-api-gateway` service (file `compose/frontend.yml`) must be changed from:

```yml
PROXY_AGGREGATED: http://ccd-data-store-api:4452
PROXY_DATA: http://ccd-data-store-api:4452
```

to, for Mac OS:

```yml
PROXY_AGGREGATED: http://docker.for.mac.localhost:4452
PROXY_DATA: http://docker.for.mac.localhost:4452
```

or to, for Windows:

```yml
PROXY_AGGREGATED: http://docker.for.win.localhost:4452
PROXY_DATA: http://docker.for.win.localhost:4452
```

The `docker.for.mac.localhost` and `docker.for.win.localhost` hostnames point to the host computer (your localhost running Docker).

For other systems, the host IP address could be used.

Once the compose files have been updated, the new configuration can be applied by running:

```
./ccd compose up -d
```

### 2. Configure local project to use containers

The local project properties must be reviewed to use the containers and comply to their configuration.

Mainly, this means:
- **Database**: pointing to the locally exposed port for the associated DB. This port used to be 5000 but has been changed to 5050 after SIDAM integration, which came to use 5000 for sidam-api application.
- **SIDAM**: pointing to the locally exposed port for SIDAM
- **S2S**:
  - pointing to the locally exposed port for `service-auth-provider-api`
  - :warning: using the right key, as defined in `service-auth-provider-api` container
- **URLs**: all URLs should be updated to point to the corresponding locally exposed port

---

## Troubleshooting

### Setup Troubleshooting

#### Networking
If you get `CCD: ERROR: Network ccd-network declared as external, but could not be found.`

Please create the network manually using docker network create ccd-network
```bash
./ccd init
```

#### Pulling images
```bash
ERROR: Get <docker_image_url>: unauthorized: authentication required
```
If you see this above authentication issue while pulling images, please follow the [Azure Setup Guide](/docs/Azure.md).

### After setup issues

#### CCD UI not loading
- it might take few minutes for all the services to startup
    > wait few minutes and then retry accessing CCD UI
- sometimes happens that some of the back-ends (data store, definition store, user profile) cannot startup because the database liquibase lock is stuck.
    > check on the back-end log if there's the following exception: 'liquibase.exception.LockException: Could not acquire change log lock'
    Execute the following command on the database:
    UPDATE DATABASECHANGELOGLOCK SET LOCKED=FALSE, LOCKGRANTED=null, LOCKEDBY=null where ID=1;
- it's possible that some of the services cannot start or crash because of lack of availabel memory. This especially when starting Idam and or ElasticSearch
    > give more memory to Docker. Configurable under Preferences -> Advanced

#### DM Store issues:

- "uk.gov.hmcts.dm.exception.AppConfigurationException: Cloub Blob Container does not exist"
    > ./bin/dm-store/document-management-store-create-blob-store-container.sh

### Idam Full Stack Troubleshooting
If running the full idam stack then see the [full stack troubleshooting section of the IDAM alt docs](./docs/IdamAlt.md#troubleshooting-full-idam-stack)

### OIDC Mismatch Troubleshooting
A recently observed error was causing the IDAM-SIM to return the SIMULATOR_OPENID_BASE_URL value instead of the SIMULATOR_OPENID_BASE_URL_OUTSIDE_NETWORK when being accessed by services outside the docker-network for the OIDC issuer.
This caused a mismatch in the OIDC issuer when running services like ccd-data-store-api outside the ccd-docker stack.
This was diagnosed by running `curl -s http://localhost:5000/o/.well-known/openid-configuration | jq -r .issuer`
Which showed that while the expected OIDC value externally was `localhost:5000` it was actually returning `idam:5000`

This was fixed by first removing all existing containers / volumes and then deleting the docker network `ccd-network`
Then by recreating from scratch.

The exact root cause is not yet known but the common precursor was updating docker desktop whilst the `ccd-network` already existed.

---

## Variables
Here are the important variables exposed in the compose files:

| Variable | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| -------- |----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| IDAM_KEY_CCD_DATA_STORE | IDAM service-to-service secret key for `ccd_data` micro-service (CCD Data store), as registered in `service-auth-provider-api`                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| IDAM_KEY_CCD_GATEWAY | IDAM service-to-service secret key for `ccd_gw` micro-service (CCD API Gateway), as registered in `service-auth-provider-api`                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| IDAM_KEY_CCD_DEFINITION_STORE | IDAM service-to-service secret key for `ccd_definition` micro-service (CCD Definition store), as registered in `service-auth-provider-api`                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| IDAM_KEY_CCD_ADMIN | IDAM service-to-service secret key for `ccd_admin` micro-service (CCD Admin Web), as registered in `service-auth-provider-api`                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| DATA_STORE_S2S_AUTHORISED_SERVICES | List of micro-service names authorised to call this service, comma-separated, as registered in `service-auth-provider-api`                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| DEFINITION_STORE_S2S_AUTHORISED_SERVICES | List of micro-services authorised to call this service, comma-separated, as registered in `service-auth-provider-api`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| USER_PROFILE_S2S_AUTHORISED_SERVICES | List of micro-services authorised to call this service, comma-separated, as registered in `service-auth-provider-api`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| DATA_STORE_TOKEN_SECRET | Secret for generation of internal event tokens                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| APPINSIGHTS_INSTRUMENTATIONKEY | Secret for Microsoft Insights logging, can be a dummy string in local                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| OIDC_ISSUER | Enforced JWT issuer value used by services that validate IDAM access tokens. This must match the token `iss` claim for the target environment and should be derived from a real token, not guessed from the public discovery URL.                                                                                                                                                                                                                                                                                                                                                                   |
| OIDC_ALLOWED_ISSUERS | Optional comma-separated additive list of exact JWT issuer values. It is passed through to services that support multi-issuer validation and should stay unset unless additional issuers are confirmed from real accepted tokens.                                                                                                                                                                                                                                                                                                                                                                      |
| CCD_CALLBACK_ALLOWED_HOSTS | Comma-separated allow-list of callback target hosts that CCD services may call over HTTPS. Local defaults include `localhost`, `127.0.0.1`, and `host.docker.internal`.                                                                                                                                                                                                                                                                                                                                                                                                                                |
| CCD_CALLBACK_ALLOWED_HTTP_HOSTS | Comma-separated allow-list of callback target hosts that CCD services may call over HTTP. Use this only when local callback endpoints are intentionally served over plain HTTP.                                                                                                                                                                                                                                                                                                                                                                                                                       |
| CCD_CALLBACK_ALLOW_PRIVATE_HOSTS | Controls whether callback targets on private or local hostnames are allowed. This supports local development callbacks to host services outside the Docker network.                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| STORAGEACCOUNT_PRIMARY_CONNECTION_STRING | (If dm-store is enabled) Secret for Azure Blob Storage. It is pointing to dockerized Azure Blob Storage emulator.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| STORAGE_CONTAINER_DOCUMENT_CONTAINER_NAME | (If dm-store is enabled) Container name for Azure Blob Storage                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| AM_DB | Access Management database name                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| AM_DB_USERNAME | Access Management database username                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| AM_DB_PASSWORD | Access Management database password                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| WIREMOCK_SERVER_MAPPINGS_PATH | Path to the WireMock mapping files. If not set, it will use the default mappings from the project repository. __Note__: If setting the variable, please keep all WireMock json stub files in a directory named _mappings_ and exclude this directory in the path. For e.g. if you place the _mappings_ in /home/user/mappings then export WIREMOCK_SERVER_MAPPINGS_PATH=/home/user. Stop the service and start service using command `./ccd compose up -d ccd-test-stub-service`. If switching back to repository mappings please unset the variable using command `unset WIREMOCK_SERVER_MAPPINGS_PATH` |
| IDAM_KEY_CASE_DOCUMENT | IDAM service-to-service secret key for `ccd_case_document_am_api` micro-service (CCD Case Document Am Api), as registered in `service-auth-provider-api`                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| IDAM_KEY_TS_TRANSLATION_SERVICE | IDAM service-to-service secret key for `ts-translation-service` micro-service (Ts Translation Service), as registered in `service-auth-provider-api`                                                                                                                                                                                                                                                                                                                                                                                                                                                     |

---

## Remarks

- A container can be configured to call a localhost host resource with the localhost shortcut added for docker containers recently. However the shortcut must be set according the docker host operating system.

```bash
# for Mac
docker.for.mac.localhost
# for Windows
docker.for.win.localhost
```

Remember that once you changed the above for a particular app you have to make sure the container configuration for that app does not try to automatically start the dependency that you have started locally. To do that either comment out the entry for the locally running app from the **depends_on** section of the config or start the app with **--no-deps** flag.

- If you happen to run `docker compose up` before setting up the environment variables, you will probably get error while starting the DB. In that
case, clear the containers but also watch out for volumes created to be cleared to get a fresh start since some initialisation scripts don't run if
you have already existing volume for the container.

```bash
$ docker volume list
DRIVER              VOLUME NAME

# better be empty
```

ccd-docker

ccd-docker Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

ccd-docker

ccd-docker Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README