Fun with SELinux on OpenShift Origin

selinux_origin

Dan Walsh did a great Youtube demo of SELinux that I like to show at my Meetups. It’s pretty easy to setup a demo environment that lets you play with the same demo. Here’s how.

First, build a RHEL 6.5 VM. I’ve been using Fedora as my primary OS at home for some time now, so KVM is my go-to hypervisor for demos. Any will work; VirtualBox is a good option for Mac and Windows users.
NB: be sure to go to virtualbox.org. The dot-com used to belong to a porn site, which, I can tell you, was embarrassing if you accidentally went to that site during a presentation. Mercifully, the site seems to be gone now.

For other KVM users, here’s how I built my VM. Instead of messing around with anaconda, I always kickstart my VMs. A tool called virt-install makes this easy.

Install the Apache web server, and open the firewall

$ sudo su -
# yum install -y httpd
(output truncated)
# firewall-cmd --enable-service httpd
success

If you want this to stick, do the following

# firewall-cmd --permanent --enable-service httpd
success

If you haven’t already generated ssh keys, do so

# ssh-keygen -N '' -t dsa -f ~/.ssh/id_dsa
Generating public/private dsa key pair.
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
52:17:f6:7d:17:58:50:ba:99:11:8a:1f:9e:c8:2f:aa root@x230
The key's randomart image is:
+--[ DSA 1024]----+
|          o  o=+ |
|         ..o.oo .|
|        ...o.o. o|
|       ...+ o =..|
|      . So + +   |
|       .  .      |
|         . .     |
|        . .      |
|     E..         |
+-----------------+
# PUB=`cat ~/.ssh/id_dsa.pub`

Be sure to load your public key into a variable that we’ll use in the kickstart script.

Now let’s build our kickstart.cfg file
Note that this config is very unsecure and only suitable for demos. Password hash is for ‘changeme’.

# cat >/var/www/html/broker.ks <<EOFKICKSTART
install
cdrom
lang en_US.UTF-8
keyboard us
network --onboot yes --device eth0 --bootproto dhcp --noipv6
timezone --utc America/New_York
rootpw  --iscrypted $6$QgevUVWY7.dTjKz6$wxw5QRKcrXxL.eoYHuBOLHnliTOq4vsbexd.cj6z8Ypd5KVnHREvs6ngyeBdO/GA7KFL7feaSYhGhBwqHELut0
selinux --enforcing
authconfig --enableshadow --passalgo=sha512
firewall --service=ssh
text
reboot

zerombr
clearpart --all
part /boot --fstype ext3 --size=500
part swap --size=512
part pv.01 --size=1 --grow
volgroup vg_root pv.01
logvol / --vgname=vg_root --size=1 --grow --name=lv_root

network --bootproto=static --ip=192.168.122.2 --netmask=255.255.255.0 --gateway=192.168.122.1 --nameserver=192.168.122.1
bootloader --location=mbr --timeout=5 --append="rhgb quiet"

%packages
@base
%end

%post

/usr/sbin/useradd -m nonroot
echo changeme | passwd --stdin nonroot
mkdir /home/nonroot/.ssh
chmod 700 /home/nonroot/.ssh
cat > /home/nonroot/.ssh/authorized_keys <<EOF_authorized_keys
${PUB}
EOF_authorized_keys
chmod 600 /home/nonroot/.ssh/authorized_keys
chown -R nonroot /home/nonroot/.ssh
chcon unconfined_u:object_r:user_home_t:s0 /home/nonroot/.ssh
chcon unconfined_u:object_r:user_home_t:s0 /home/nonroot/.ssh/authorized_keys

cat >> /etc/sudoers <<EOF_sudoers
nonroot ALL=(ALL) NOPASSWD: ALL
EOF_sudoers

cat > /etc/sysconfig/iptables <<EOF_iptables
# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT
EOF_iptables

echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
EOFKICKSTART

Add our VM to /etc/hosts

# echo "192.168.122.2 broker.example.com broker" >> /etc/hosts

We’ll have to make the RHEL 6.5 media available via http.

# mount -o loop /path/to/rhel.iso /var/www/html/rhel65

Optional: setup a new storage pool
Anaconda auto-partitioning puts most of your storage space in /home, which is why I like an alternate location.

# mkdir -p /home/libvirt/images
# cat >./home-libvirt.xml <<EOF
<pool type='dir'>
  <name>home-libvirt</name>
  <source>
  </source>
  <target>
    <path>/home/libvirt/images</path>
    <permissions>
      <mode>0755</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</pool>
# virsh pool-create home-libvirt.xml

Now we’re ready to fire up the VM and kickstart it.

# virt-install 
  --name broker 
  --hvm 
  --ram 4096 
  --disk path=/home/libvirt/images/broker.img,size=16,format=qcow2 
  --network network:default 
  --vnc 
  --connect qemu:///system 
  --os-type=linux 
  --os-variant=rhel6 
  --location http://192.168.122.1/rhel65/ 
  -x "ks=http://192.168.122.1/broker.ks"

Everything should work automagically from here. A virt-viewer window will pop up, displaying the console via the Spice protocol. Once your VM is up, ssh into it, or login from the console.

oo-install: The OpenShift Origin installer
Use this procedure to get setup via oo-install. Normally oo-install would prompt you for input, but we’re going to pre-populate a config file with most things already filled-in.

# hostname broker.example.com
# sed -i -e 's/HOSTNAME=.*/HOSTNAME=broker.example.com/' /etc/sysconfig/network
# MYIP=`ifconfig eth0 | grep ^ *inet | awk '{print $2}' | cut -d: -f2`
# echo "${MYIP} broker.example.com broker" >> /etc/hosts
# yum install -y http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
# cat > /etc/yum.repos.d/openshift-origin-deps.repo <<EOF
[openshift-origin-deps]
name=OpenShift Origin Dependencies - EL6
baseurl=http://mirror.openshift.com/pub/origin-server/release/3/rhel-6/dependencies/$basearch/
gpgcheck=0
EOF

Attach your system to RHN
I do this with an activation key I made, you could also use subscription-manager.

# rhnreg_ks --activationkey=1-mykey

Install the prereqs and configure oo-install

# yum install -y ruby unzip curl bind httpd-tools ruby193-ruby puppet augeas
# mkdir ~/.openshift
# cat >~/.openshift/oo-install-cfg.yml <<EOF
---
Version: 0.0.1
Subscription:
  type: yum
  repos_base: https://mirror.openshift.com/pub/origin-server/release/3/rhel-6
  jenkins_repo_base: http://pkg.jenkins-ci.org/redhat
Name: OpenShift Installer Configuration
Deployment:
  Hosts:
  - host: broker.example.com
    roles:
    - msgserver
    - dbserver
    - broker
    - node
    ssh_host: localhost
    user: root
    ip_addr: 192.168.122.2
    named_ip_addr: 192.168.122.2
    ip_interface: eth0
    state: new
  DNS:
    app_domain: example.com
    register_components: Y
    component_domain: example.com
Vendor: OpenShift Origin Community
Description: This is the configuration file for the OpenShift Installer.
EOF

Now, run the OpenShift Origin installer, oo-install

# sh <(curl -s https://install.openshift.com/)
Checking for necessary tools...
...looks good.
Downloading oo-install package...
Extracting oo-install to temporary directory...
Starting oo-install...
OpenShift Installer (Build 20140122-2029)
----------------------------------------------------------------------

Welcome to OpenShift.

This installer will guide you through a basic system deployment, based
on one of the scenarios below.

Select from the following installation scenarios.
You can also type '?' for Help or 'q' to Quit:
1. Install OpenShift Origin
2. Add a Node to an OpenShift Origin deployment
Type a selection and press <return>: 1


Here are the details of your current deployment.

Note: ActiveMQ, MongoDB and named will all be installed on the Broker.
For more flexibility, rerun the installer in advanced mode (-a).

DNS Settings
  * App Domain: example.com
  * Register OpenShift components with OpenShift DNS? Yes
  * Component Domain: example.com

Role Assignments
+--------+--------------------+
| Broker | broker.example.com |
| Node   | broker.example.com |
+--------+--------------------+

Host Information
+----------------+--------------------+
| Host           | broker.example.com |
| Roles          | Broker, Node       |
| SSH Host       | localhost          |
| User           | root               |
| IP Addr        | 192.168.122.2      |
| BIND DNS Addr  | 192.168.122.2      |
| IP Interface   | eth0               |
| Install Status | new                |
+----------------+--------------------+

Do you want to change the deployment info? (y/n/q/?) n

Here is the subscription configuration that the installer will use for
this deployment.
+-------------------+-----------------------------------------------------------------+
| Setting           | Value                                                           |
+-------------------+-----------------------------------------------------------------+
| type              | yum                                                             |
| repos_base        | https://mirror.openshift.com/pub/origin-server/release/3/rhel-6 |
| jboss_repo_base   | -                                                               |
| jenkins_repo_base | http://pkg.jenkins-ci.org/redhat                                |
| os_repo           | -                                                               |
| os_optional_repo  | -                                                               |
+-------------------+-----------------------------------------------------------------+

Do you want to make any changes to the subscription info in the
configuration file? (y/n/q/?) n

Do you want to set any temporary subscription settings for this
installation only? (y/n/q/?) n

This will run for a bit as your single-node OpenShift deployment is built.
Reboot, then we’re ready to start playing.

# shutdown -r now

You can do everything from the web console (http://broker.example.com/console), but since we’ve been all CLI so far, let’s stick with that.

Now I use my regular user to connect to OpenShift Online, where this blog is actually hosted. So let’s make a demo user to protect our regular account’s rhc configuration.
On your host system, create a demo user, and ensure the rhc client is installed

# useradd -m demo
# yum install -y rhc
# su - demo
$ rhc setup --server=broker.example.com
OpenShift Client Tools (RHC) Setup Wizard

This wizard will help you upload your SSH keys, set your application namespace, and check that other programs like Git
are properly installed.

The server's certificate is self-signed, which means that a secure connection can't be established to
'broker.example.com'.

You may bypass this check, but any data you send to the server could be intercepted by others.

Connect without checking the certificate? (yes|no): yes
Login to broker.example.com: demo
Password: ********

OpenShift can create and store a token on disk which allows to you to access the server without using your password. The
key is stored in your home directory and should be kept secret.  You can delete the key at any time by running 'rhc
logout'.
Generate a token now? (yes|no) yes
Generating an authorization token for this client ... lasts about 23 hours

Saving configuration to /home/demo/.openshift/express.conf ... done

Checking for git ... found git version 1.8.4.2

Checking common problems .. done

Checking for a domain ... demons

Checking for applications ... none

Run 'rhc create-app' to create your first application.

  Do-It-Yourself 0.1 rhc create-app <app name> diy-0.1
  Jenkins Server     rhc create-app <app name> jenkins-1
  Node.js 0.10       rhc create-app <app name> nodejs-0.10
  Node.js 0.6        rhc create-app <app name> nodejs-0.6
  PHP 5.3            rhc create-app <app name> php-5.3
  Perl 5.10          rhc create-app <app name> perl-5.10
  Python 2.6         rhc create-app <app name> python-2.6
  Python 2.7         rhc create-app <app name> python-2.7
  Python 3.3         rhc create-app <app name> python-3.3
  Ruby 1.8           rhc create-app <app name> ruby-1.8
  Ruby 1.9           rhc create-app <app name> ruby-1.9

  You are using 0 of 100 total gears
  The following gear sizes are available to you: small

Your client tools are now configured.

Note that the default demo account in OpenShift is username ‘demo’, password ‘changeme’.

Create a domain and two apps
I’ve been using ‘demons’ for demo-namespace, not the opposite of angels.

# sed -i -e '/^192.168.122.2/s/$/ php-demons.example.com python-demons.example.com/' /etc/hosts
# su - demo
$ rhc domain create demons
Creating domain 'demons' ... done
You may now create an application using the 'rhc create-app' command
rhc app create php php-5.3 -n demons
Application Options
-------------------
Domain:     demons
Cartridges: php-5.3
Gear Size:  default
Scaling:    no

Creating application 'php' ... done


Waiting for your DNS name to be available ... done

Cloning into 'php'...

Your application 'php' is now available.

  URL:        http://php-demons.example.com/
  SSH to:     52e14cdd6892dfafd8000003@php-demons.example.com
  Git remote: ssh://52e14cdd6892dfafd8000003@php-demons.example.com/~/git/php.git/
  Cloned to:  /home/demo/php

Run 'rhc show-app php' for more details about your app.
$ rhc app create python python-2.7 -n demons
Application Options
-------------------
Domain:     demons
Cartridges: python-2.7
Gear Size:  default
Scaling:    no

Creating application 'python' ... done


Waiting for your DNS name to be available ... done

Cloning into 'python'...
The authenticity of host 'python-demons.example.com (192.168.122.2)' can't be established.
RSA key fingerprint is b0:96:03:d9:d2:dd:b3:a8:14:22:a6:72:8a:65:40:f5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'python-demons.example.com' (RSA) to the list of known hosts.

Your application 'python' is now available.

  URL:        http://python-demons.example.com/
  SSH to:     52e14cfd6892dfafd800001e@python-demons.example.com
  Git remote: ssh://52e14cfd6892dfafd800001e@python-demons.example.com/~/git/python.git/
  Cloned to:  /home/demo/python

Run 'rhc show-app python' for more details about your app.

Ok, now our apps exist, let’s ssh into one and see what SELinux context we end up with.

$ rhc ssh php
Connecting to 52e14cdd6892dfafd8000003@php-demons.example.com ...

    *********************************************************************

    You are accessing a service that is for use only by authorized users.  
    If you do not have authorization, discontinue use at once. 
    Any use of the services is subject to the applicable terms of the 
    agreement which can be found at: 
    https://www.openshift.com/legal

    *********************************************************************

    Welcome to OpenShift shell

    This shell will assist you in managing OpenShift applications.

    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!
    Shell access is quite powerful and it is possible for you to
    accidentally damage your application.  Proceed with care!
    If worse comes to worst, destroy your application with "rhc app delete"
    and recreate it
    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!

    Type "help" for more info.


[php-demons.example.com 52e14cdd6892dfafd8000003]> id -Z
unconfined_u:system_r:openshift_t:s0:c0,c1000

Now we’re into the meat of Dan’s SELinux demo. We can see OpenShift’s Multi-Category Security (MCS) at work. This gear’s context has a category of c0,c1000. Out of curiosity, let’s check out the other gear.

[php-demons.example.com 52e14cdd6892dfafd8000003]> exit
exit
Connection to php-demons.example.com closed.
$ rhc ssh python
Connecting to 52e14cfd6892dfafd800001e@python-demons.example.com ...

    *********************************************************************

    You are accessing a service that is for use only by authorized users.  
    If you do not have authorization, discontinue use at once. 
    Any use of the services is subject to the applicable terms of the 
    agreement which can be found at: 
    https://www.openshift.com/legal

    *********************************************************************

    Welcome to OpenShift shell

    This shell will assist you in managing OpenShift applications.

    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!
    Shell access is quite powerful and it is possible for you to
    accidentally damage your application.  Proceed with care!
    If worse comes to worst, destroy your application with "rhc app delete"
    and recreate it
    !!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!

    Type "help" for more info.


[python-demons.example.com 52e14cfd6892dfafd800001e]> id -Z
unconfined_u:system_r:openshift_t:s0:c0,c1001

Ok, so the other gear has a different SELinux context, namely its category, which is c0,c1001.

This is where things get interesting. OpenShift lets users add DIY cartridges that can contain anything — potentially even malicious code. But OpenShift “fails well,” so it’s safe to allow that. Let’s pretend we’ve uploaded a DIY cartridge that contains super 1337 hax0r malicious stuff that immediately gets us a privilege escalation to UID/GID 0. We’re going to model that by ssh-ing into the broker/node as root, then setting our context to that of our rooted gear, or in this case, php.

Copy the php gear’s SELinux context to the clipboard, “unconfined_u:system_r:openshift_t:s0:c0,c1000”
We’re starting from our host system.

$ ssh nonroot@broker
Last login: Thu Jan 23 11:24:51 2014 from 192.168.122.1
[nonroot@broker ~]$ sudo su -
[root@broker ~]# id -a
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[root@broker ~]# runcon unconfined_u:system_r:openshift_t:s0:c0,c1000 bash
bash: /root/.bashrc: Permission denied
bash-4.1# id -a
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:system_r:openshift_t:s0:c0,c1000

We’re still root, but instead of having the full range of categories (ranges are denoted by a ‘.’), or c0.c1023, we have a discrete category, c0,c1000.

Now we’ll try and do some naughty stuff.

bash-4.1# touch /virus
touch: cannot touch `/virus': Permission denied
bash-4.1# cat /etc/shadow
cat: /etc/shadow: Permission denied
bash-4.1# cd /var/lib/openshift/
bash-4.1# ls
52e14cdd6892dfafd8000003  52e14cfd6892dfafd800001e
bash-4.1# cd 52e14cfd6892dfafd800001e/
bash: cd: 52e14cfd6892dfafd800001e/: Permission denied

We’re not able to mess with the system, or even see what’s in the other gear, ‘python’.

OpenShift is very secure. It fails well. Even if a gear gets root, there’s very little they can do to hurt the system.

Once again, thanks to Dan Walsh for the original demo.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s