Conjure-up with MAAS

Conjure-up a K8s Multinode Cluster on Ubuntu 18.04 LTS

 Posted on May 24, 2019  |  11 minutes  |  2269 words  |  blinkeye

Conjure-up

Follow https://jaas.ai/canonical-kubernetes.

Start with:

conjure-up kubernetes-core --debug

Currently the spell kubernetes-core breaks, see conjure-up/issues/1600.

Still, go through the GUI configuration, watch the log .cache/conjure-up/conjure-up.log

2019-05-18 20:29:03,689 [DEBUG] conjure-up/kubernetes-core - juju.py:252 - bootstrap cmd: ['/snap/bin/juju', 'bootstrap', 'cloud-maas-569', 'conjure-up-cloud-maas-569-d31', '--default-model', 'conjure-kubernetes-core-272', '--config', 'image-stream=daily', '--credential', 'conjure-cloud-maas-569-886']

and take the command:

/snap/bin/juju bootstrap cloud-maas-65e conjure-up-cloud-maas-65e-f30 --default-model conjure-kubernetes-core-9b1 --config image-stream=daily --credential conjure-cloud-maas-65e-8b0

Use:

juju clouds

Clouds on controller "conjure-up-cloud-maas-9c4-ba7":

Cloud           Regions  Default  Type  Description
cloud-maas-9c4        0           maas

and continue deployment with:

sudo snap install juju --classic
juju deploy --dry-run kubernetes-core
juju deploy kubernetes-core

NOTE: This uses outdated revisions for all components:

blinkeye@braindump:~$ juju status

Model                        Controller                     Cloud/Region    Version  SLA          Timestamp
conjure-kubernetes-core-096  conjure-up-cloud-maas-9c4-ba7  cloud-maas-9c4  2.6.2    unsupported  09:38:56+02:00

App                Version  Status  Scale  Charm              Store       Rev  OS      Notes
easyrsa            3.0.1    active      1  easyrsa            jujucharms  235  ubuntu  
etcd               3.2.10   active      1  etcd               jujucharms  415  ubuntu  
flannel            0.10.0   active      2  flannel            jujucharms  404  ubuntu  
kubernetes-master  1.14.1   active      1  kubernetes-master  jujucharms  654  ubuntu  exposed
kubernetes-worker  1.14.1   active      1  kubernetes-worker  jujucharms  519  ubuntu  exposed

Unit                  Workload  Agent  Machine  Public address  Ports           Message
easyrsa/0*            active    idle   0/lxd/0  10.10.11.222                    Certificate Authority connected.
etcd/0*               active    idle   0        10.10.11.220    2379/tcp        Healthy with 1 known peer
kubernetes-master/0*  active    idle   0        10.10.11.220    6443/tcp        Kubernetes master running.
  flannel/1           active    idle            10.10.11.220                    Flannel subnet 10.1.89.1/24
kubernetes-worker/0*  active    idle   1        10.10.11.221    80/tcp,443/tcp  Kubernetes worker running.
  flannel/0*          active    idle            10.10.11.221                    Flannel subnet 10.1.22.1/24

Machine  State    DNS           Inst id              Series  AZ       Message
0        started  10.10.11.220  square-wolf          bionic  default  Deployed
0/lxd/0  started  10.10.11.222  juju-2b44a0-0-lxd-0  bionic  default  Container started
1        started  10.10.11.221  modest-cobra         bionic  default  Deployed

Better use:

juju deploy --dry-run cs:~containers/bundle/kubernetes-core-681

Upgrade

upgrade-model

For unknown reason, the upgrade-model does not work:

juju upgrade-model --dry-run
no upgrades available

upgrade-channel

Upgrade to latest Charm version (channel edge):

juju config kubernetes-master channel=1.14/edge
juju config kubernetes-worker channel=1.14/edge

then execute manual upgrade:

juju run-action kubernetes-master/0 upgrade  
juju run-action kubernetes-worker/0 upgrade

this upgrades to kubernetes-master:654 and kubernetes-worker:519:

blinkeye@braindump:~$ juju status

Model                        Controller                     Cloud/Region    Version  SLA          Timestamp
conjure-kubernetes-core-096  conjure-up-cloud-maas-9c4-ba7  cloud-maas-9c4  2.6.2    unsupported  09:58:39+02:00

App                Version  Status  Scale  Charm              Store       Rev  OS      Notes
easyrsa            3.0.1    active      1  easyrsa            jujucharms  235  ubuntu  
etcd               3.2.10   active      1  etcd               jujucharms  415  ubuntu  
flannel            0.10.0   active      2  flannel            jujucharms  404  ubuntu  
kubernetes-master  1.14.2   active      1  kubernetes-master  jujucharms  654  ubuntu  exposed
kubernetes-worker  1.14.2   active      1  kubernetes-worker  jujucharms  519  ubuntu  exposed

Unit                  Workload  Agent  Machine  Public address  Ports           Message
easyrsa/0*            active    idle   0/lxd/0  10.10.11.222                    Certificate Authority connected.
etcd/0*               active    idle   0        10.10.11.220    2379/tcp        Healthy with 1 known peer
kubernetes-master/0*  active    idle   0        10.10.11.220    6443/tcp        Kubernetes master running.
  flannel/1           active    idle            10.10.11.220                    Flannel subnet 10.1.89.1/24
kubernetes-worker/0*  active    idle   1        10.10.11.221    80/tcp,443/tcp  Kubernetes worker running.
  flannel/0*          active    idle            10.10.11.221                    Flannel subnet 10.1.22.1/24

Machine  State    DNS           Inst id              Series  AZ       Message
0        started  10.10.11.220  square-wolf          bionic  default  Deployed
0/lxd/0  started  10.10.11.222  juju-2b44a0-0-lxd-0  bionic  default  Container started
1        started  10.10.11.221  modest-cobra         bionic  default  Deployed

upgrade-charm

Double check against the latest version for:

  • kubernetes-core: 677
  • kubernetes-master: 677
  • kubernetes-client: 537
  • easyrsa: 247
  • etcd: 425
  • flannel: 413

Upgrade:

juju upgrade-charm kubernetes-master --revision 677
juju upgrade-charm kubernetes-worker --revision 535
juju upgrade-charm easyrsa --revision 247
juju upgrade-charm etcd --revision 425
juju upgrade-charm flannel --revision 413

which results in:

blinkeye@braindump:~$ juju status

Model                        Controller                     Cloud/Region    Version  SLA          Timestamp
conjure-kubernetes-core-096  conjure-up-cloud-maas-9c4-ba7  cloud-maas-9c4  2.6.2    unsupported  10:18:52+02:00

App                Version  Status  Scale  Charm              Store       Rev  OS      Notes
easyrsa            3.0.1    active      1  easyrsa            jujucharms  247  ubuntu  
etcd               3.2.10   active      1  etcd               jujucharms  425  ubuntu  
flannel            0.10.0   active      2  flannel            jujucharms  413  ubuntu  
kubernetes-master  1.14.2   active      1  kubernetes-master  jujucharms  677  ubuntu  exposed
kubernetes-worker  1.14.2   active      1  kubernetes-worker  jujucharms  535  ubuntu  exposed

Unit                  Workload  Agent  Machine  Public address  Ports           Message
easyrsa/0*            active    idle   0/lxd/0  10.10.11.222                    Certificate Authority connected.
etcd/0*               active    idle   0        10.10.11.220    2379/tcp        Healthy with 1 known peer
kubernetes-master/0*  active    idle   0        10.10.11.220    6443/tcp        Kubernetes master running.
  flannel/1           active    idle            10.10.11.220                    Flannel subnet 10.1.89.1/24
kubernetes-worker/0*  active    idle   1        10.10.11.221    80/tcp,443/tcp  Kubernetes worker running.
  flannel/0*          active    idle            10.10.11.221                    Flannel subnet 10.1.22.1/24

Machine  State    DNS           Inst id              Series  AZ       Message
0        started  10.10.11.220  square-wolf          bionic  default  Deployed
0/lxd/0  started  10.10.11.222  juju-2b44a0-0-lxd-0  bionic  default  Container started
1        started  10.10.11.221  modest-cobra         bionic  default  Deployed

juju gui

Start the gui:

juju gui

GUI 2.14.0 for model "admin/conjure-kubernetes-core-096" is enabled at:
  https://10.10.11.219:17070/gui/u/admin/conjure-kubernetes-core-096
Your login credential is:
  username: admin
  password: 2c3f093575c0987ccca78e6e58fb301e

then login and export/import the configuration:

series: bionic
applications:
  kubernetes-worker:
    charm: 'cs:~containers/kubernetes-worker-535'
    num_units: 1
    options:
      channel: 1.14/edge
    constraints: cores=4 mem=4096 root-disk=16384
    expose: true
    series: bionic
    annotations:
      gui-x: '100'
      gui-y: '850'
    to:
      - '1'
  etcd:
    charm: 'cs:~containers/etcd-425'
    num_units: 1
    options:
      channel: 3.2/stable
    series: bionic
    annotations:
      gui-x: '800'
      gui-y: '550'
    to:
      - '0'
  easyrsa:
    charm: 'cs:~containers/easyrsa-247'
    num_units: 1
    series: bionic
    annotations:
      gui-x: '450'
      gui-y: '550'
    to:
      - 'lxd:etcd/0'
  flannel:
    charm: 'cs:~containers/flannel-413'
    series: bionic
    annotations:
      gui-x: '450'
      gui-y: '750'
  kubernetes-master:
    charm: 'cs:~containers/kubernetes-master-677'
    num_units: 1
    options:
      channel: 1.14/edge
    constraints: cores=2 mem=4096 root-disk=16384
    expose: true
    series: bionic
    annotations:
      gui-x: '800'
      gui-y: '850'
    to:
      - '0'
relations:
  - - 'kubernetes-master:kube-api-endpoint'
    - 'kubernetes-worker:kube-api-endpoint'
  - - 'kubernetes-master:kube-control'
    - 'kubernetes-worker:kube-control'
  - - 'kubernetes-master:certificates'
    - 'easyrsa:client'
  - - 'kubernetes-master:etcd'
    - 'etcd:db'
  - - 'kubernetes-worker:certificates'
    - 'easyrsa:client'
  - - 'etcd:certificates'
    - 'easyrsa:client'
  - - 'flannel:etcd'
    - 'etcd:db'
  - - 'flannel:cni'
    - 'kubernetes-master:cni'
  - - 'flannel:cni'
    - 'kubernetes-worker:cni'
machines:
  '0':
    series: bionic
    constraints: arch=amd64 cpu-cores=2 mem=4096
  '1':
    series: bionic
    constraints: arch=amd64 cpu-cores=4 mem=4096

Kubernetes test with microbot

$ juju run-action kubernetes-worker/0 microbot replicas=5
   Action queued with id: db7cc72b-5f35-4a4d-877c-284c4b776eb8
    
$ juju show-action-output db7cc72b-5f35-4a4d-877c-284c4b776eb8
    ...
    http://microbot.10.10.11.221.xip.io

go to http://microbot.10.10.11.221.xip.io and hit refresh (round robins to different pod).

Remove with:

juju run-action kubernetes-worker/0 microbot delete=true

kubernetes-core

Alternatively, go to: https://jaas.ai/u/containers/kubernetes-core/bundle/681:

juju deploy --dry-run cs:~containers/bundle/kubernetes-core-681

Located bundle "cs:~containers/bundle/kubernetes-core-681"
Resolving charm: cs:~containers/easyrsa-241
Resolving charm: cs:~containers/etcd-423
Resolving charm: cs:~containers/flannel-408
Resolving charm: cs:~containers/kubernetes-master-671
Resolving charm: cs:~containers/kubernetes-worker-530
Changes to deploy bundle:
- upload charm cs:~containers/easyrsa-241 for series bionic
- upgrade easyrsa to use charm cs:~containers/easyrsa-241 for series bionic
- set annotations for easyrsa
- upload charm cs:~containers/etcd-423 for series bionic
- upgrade etcd to use charm cs:~containers/etcd-423 for series bionic
- set annotations for etcd
- upload charm cs:~containers/flannel-408 for series bionic
- upgrade flannel to use charm cs:~containers/flannel-408 for series bionic
- set annotations for flannel
- upload charm cs:~containers/kubernetes-master-671 for series bionic
- upgrade kubernetes-master to use charm cs:~containers/kubernetes-master-671 for series bionic
- set application options for kubernetes-master
- set annotations for kubernetes-master
- upload charm cs:~containers/kubernetes-worker-530 for series bionic
- upgrade kubernetes-worker to use charm cs:~containers/kubernetes-worker-530 for series bionic
- set application options for kubernetes-worker
- set annotations for kubernetes-worker

Scaling

kubernetes-worker

juju add-unit kubernetes-worker

Remove machine

juju remove-machine 5
juju remove-machine 5 --force

Storage

Follow: https://medium.com/@knobby/nfs-default-storage-in-kubernetes-with-cdk-847336cc4a72

Juju

Instructions to upgrade applications.

Show current status and application versions:

$ juju status

Model                        Controller                     Cloud/Region    Version  SLA          Timestamp
conjure-kubernetes-core-096  conjure-up-cloud-maas-9c4-ba7  cloud-maas-9c4  2.6.2    unsupported  09:45:46+02:00

App                Version  Status  Scale  Charm              Store       Rev  OS      Notes
easyrsa            3.0.1    active      1  easyrsa            jujucharms  247  ubuntu  
etcd               3.2.10   active      1  etcd               jujucharms  425  ubuntu  
flannel            0.10.0   active      5  flannel            jujucharms  413  ubuntu  
kubernetes-master  1.14.1   active      1  kubernetes-master  jujucharms  681  ubuntu  exposed
kubernetes-worker  1.14.2   active      4  kubernetes-worker  jujucharms  535  ubuntu  exposed
nfs                         active      1  nfs                jujucharms    9  ubuntu  

Unit                  Workload  Agent  Machine  Public address  Ports           Message
easyrsa/0*            active    idle   0/lxd/0  10.10.11.222                    Certificate Authority connected.
etcd/0*               active    idle   0        10.10.11.220    2379/tcp        Healthy with 1 known peer
kubernetes-master/0*  active    idle   0        10.10.11.220    6443/tcp        Kubernetes master running.
  flannel/1*          active    idle            10.10.11.220                    Flannel subnet 10.1.89.1/24
kubernetes-worker/0*  active    idle   1        10.10.11.221    80/tcp,443/tcp  Kubernetes worker running.
  flannel/0           active    idle            10.10.11.221                    Flannel subnet 10.1.44.1/24
kubernetes-worker/2   active    idle   4        10.10.11.225    80/tcp,443/tcp  Kubernetes worker running.
  flannel/3           active    idle            10.10.11.225                    Flannel subnet 10.1.23.1/24
kubernetes-worker/6   active    idle   8        10.10.11.229    80/tcp,443/tcp  Kubernetes worker running.
  flannel/7           active    idle            10.10.11.229                    Flannel subnet 10.1.98.1/24
kubernetes-worker/7   active    idle   9        10.10.11.230    80/tcp,443/tcp  Kubernetes worker running.
  flannel/8           active    idle            10.10.11.230                    Flannel subnet 10.1.88.1/24
nfs/0*                active    idle   2        10.10.11.223                    NFS ready

Machine  State    DNS           Inst id              Series  AZ       Message
0        started  10.10.11.220  square-wolf          bionic  default  Deployed
0/lxd/0  started  10.10.11.222  juju-2b44a0-0-lxd-0  bionic  default  Container started
1        started  10.10.11.221  modest-cobra         bionic  default  Deployed
2        started  10.10.11.223  casual-mammal        bionic  default  Deployed
4        started  10.10.11.225  nuc2                 bionic  default  Deployed
8        started  10.10.11.229  nuc1                 bionic  default  Deployed
9        started  10.10.11.230  nuc3                 bionic  default  Deployed

Juju GUI

Get ip and port number from the api-endpoints field and username and password from the account details:

$ juju show-controller --show-password

conjure-up-cloud-maas-9c4-ba7:
  details:
    uuid: f5c53ab4-0804-4fdc-8ded-0d8a0d237a3b
    controller-uuid: f5c53ab4-0804-4fdc-8ded-0d8a0d237a3b
    api-endpoints: ['10.10.11.219:17070']
    cloud: cloud-maas-9c4
    agent-version: 2.6.2
    mongo-version: 3.6.3
    ...
  controller-machines:
    "0":
      instance-id: 7kyryq
  models:
    conjure-kubernetes-core-096:
      uuid: 49ff2929-67cd-4fa4-862d-caa1002b44a0
      model-uuid: 49ff2929-67cd-4fa4-862d-caa1002b44a0
      machine-count: 7
      core-count: 31
    controller:
      uuid: c3bf8540-a24f-4e4b-8abd-98be08a93dbc
      model-uuid: c3bf8540-a24f-4e4b-8abd-98be08a93dbc
      machine-count: 1
      core-count: 1
  current-model: admin/conjure-kubernetes-core-096
  account:
    user: admin
    access: superuser
    password: 2c3f093575c0987ccca78e6e58fb301e
/img/juju/login.webp

GUI login

Upgrade application

For CLI commands see this link. In short:

$ juju upgrade-charm kubernetes-master --channel=stable
$ juju upgrade-charm kubernetes-worker --channel=stable
$ juju upgrade-charm flannel --channel=stable
$ juju upgrade-charm easyrsa --channel=stable
$ juju upgrade-charm etcd --channel=stable

or all together:

for charm in etcd easyrsa flannel; do juju upgrade-charm $charm --channel=stable; done

Watch status/progress with:

$ watch -n1 -c juju status --color

Upgrade kubernetes-worker:

/img/juju/upgrade_gui1a.webp

Select component

/img/juju/upgrade_gui1.webp

Choose (new) target version

SSH login

$ juju ssh -m controller 0
$ tail -f /var/log/juju/*.log

SSH login without Juju

The Juju SSH key is in:

$HOME/.local/share/juju/ssh/juju_id_rsa

use it to log in to the controller for example:

blinkeye@braindump:~$ ssh -i /home/blinkeye/.local/share/juju/ssh/juju_id_rsa ubuntu@10.10.11.219

derived from:

$ juju ssh 1
$ ps axuf | grep juju
blinkeye  2666  0.0  0.0  15180  5192 pts/1    S+   09:56   0:00  |   \_ watch -n1 -c juju status --color
blinkeye  2667  0.0  0.0  15180  2624 pts/1    S+   09:56   0:00  |       \_ watch -n1 -c juju status --color
blinkeye  2668  0.0  0.0   4624   784 pts/1    S+   09:56   0:00  |           \_ sh -c juju status --color
blinkeye  2669  0.0  0.1 1364740 57348 pts/1   Sl+  09:56   0:00  |               \_ /snap/juju/8049/bin/juju status --color
blinkeye 16762  0.0  0.0  13132  1020 pts/4    S+   10:16   0:00  |   \_ grep --color=auto juju
blinkeye 28665  0.0  0.1 1807676 56320 pts/8   Sl+  Jun08   0:39  |   \_ /snap/juju/8049/bin/juju ssh 1
blinkeye 28704  0.0  0.0  46840  5936 pts/8    S+   Jun08   0:01  |       \_ ssh -o StrictHostKeyChecking yes -o PasswordAuthentication no -o ServerAliveInterval 30 -t -t -o UserKnownHostsFile /tmp/ssh_known_hosts841436788 -i /home/blinkeye/.local/share/juju/ssh/juju_id_rsa ubuntu@10.10.11.221
blinkeye 28302  0.0  0.1 1808188 56164 pts/3   Sl+  Jun08   0:39  |   \_ /snap/juju/8049/bin/juju ssh 4
blinkeye 28342  0.0  0.0  46924  6412 pts/3    S+   Jun08   0:02  |       \_ ssh -o StrictHostKeyChecking yes -o PasswordAuthentication no -o ServerAliveInterval 30 -t -t -o UserKnownHostsFile /tmp/ssh_known_hosts609289403 -i /home/blinkeye/.local/share/juju/ssh/juju_id_rsa ubuntu@10.10.11.225
blinkeye 28502  0.0  0.1 1734200 55968 pts/11  Sl+  Jun08   0:39  |   \_ /snap/juju/8049/bin/juju ssh 8
blinkeye 28541  0.0  0.0  46916  6368 pts/11   S+   Jun08   0:01  |       \_ ssh -o StrictHostKeyChecking yes -o PasswordAuthentication no -o ServerAliveInterval 30 -t -t -o UserKnownHostsFile /tmp/ssh_known_hosts137775778 -i /home/blinkeye/.local/share/juju/ssh/juju_id_rsa ubuntu@10.10.11.229
blinkeye 32009  0.0  0.1 1512492 55884 pts/12  Sl+  Jun08   0:36  |   \_ /snap/juju/8049/bin/juju ssh 9
blinkeye 32046  0.0  0.0  46972  5772 pts/12   S+   Jun08   0:01  |       \_ ssh -o StrictHostKeyChecking yes -o PasswordAuthentication no -o ServerAliveInterval 30 -t -t -o UserKnownHostsFile /tmp/ssh_known_hosts069169642 -i /home/blinkeye/.local/share/juju/ssh/juju_id_rsa ubuntu@10.10.11.230

Controller maintenance

If the controller is unreachable, ssh login:

$ ssh -i /home/blinkeye/.local/share/juju/ssh/juju_id_rsa ubuntu@10.10.11.219
$ ubuntu@sought-dove:~$ dmesg 

[    7.309795] random: 7 urandom warning(s) missed due to ratelimiting
[  316.385817] systemd-journald[366]: Failed to create new system journal: No space left on device
[  316.388526] systemd-journald[366]: Failed to open system journal: No space left on device
[  316.394405] systemd-journald[366]: Failed to open system journal: No space left on device
[  316.394436] systemd-journald[366]: Failed to open system journal: No space left on device
[  316.394575] systemd-journald[366]: Failed to open system journal: No space left on device
[  316.394605] systemd-journald[366]: Failed to open system journal: No space left on device
[  316.394707] systemd-journald[366]: Failed to open system journal: No space left on device
[  316.394737] systemd-journald[366]: Failed to open system journal: No space left on device
[  316.395966] systemd-journald[366]: Failed to open system journal: No space left on device
[  316.395997] systemd-journald[366]: Failed to open system journal: No space left on device

resize qemu image:

root@braindump:/var/lib/libvirt/images# qemu-img resize 61064b54-b45e-484d-a1b2-4f8db93e9b33 +16G

WARNING: Image format was not specified for '61064b54-b45e-484d-a1b2-4f8db93e9b33' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
Image resized.

Boot controller again and resize partition by re-creating partition table:

ubuntu@sought-dove:~$ sudo fdisk /dev/vda

Welcome to fdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/vda: 23.5 GiB, 25179869184 bytes, 49179432 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x1ec85f8e

Device     Boot Start      End  Sectors  Size Id Type
/dev/vda1        2048 15624966 15622919  7.5G 83 Linux

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 
First sector (2048-49179431, default 2048): 
Last sector, +sectors or +size{K,M,G,T,P} (2048-49179431, default 49179431): 

Created a new partition 1 of type 'Linux' and of size 23.5 GiB.
Partition #1 contains a ext4 signature.

Do you want to remove the signature? [Y]es/[N]o: N 

Command (m for help): p

Disk /dev/vda: 23.5 GiB, 25179869184 bytes, 49179432 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x1ec85f8e

Device     Boot Start      End  Sectors  Size Id Type
/dev/vda1        2048 49179431 49177384 23.5G 83 Linux

Command (m for help): w
The partition table has been altered.
Syncing disks.

next, grow ext4 filesystem:

$ sudo resize2fs /dev/vda1
  resize2fs 1.44.1 (24-Mar-2018)
  Filesystem at /dev/vda1 is mounted on /; on-line resizing required
  old_desc_blocks = 1, new_desc_blocks = 3
  The filesystem on /dev/vda1 is now 6147173 (4k) blocks long.

$ sudo reboot
Canonical  Ubuntu  Cloud  Juju 

See also