Introducing the Fedora Red Team

Last week my colleagues and I started a new Special Interest Group under the Fedora Project: the Fedora Red Team.

The Fedora Red Team’s goal is to become Red Hat’s upstream cybersecurity community. But what does that actually mean?

“Cyber” is a fascinating term with a colorful history. I’ve written about it before, but the punchline is that we owe its ubiquity to William Gibson’s Neuromancer and an obscure US Government paper from the late 90s, referred to as The Marsh Report.

The context of The Marsh Report seems to have stuck — cyber has come to refer to the high-stakes interplay of offensive and defensive infosec, often at nation-state scale. If you want to get ontological, you could say “cyber” includes the three sub-domains of Computer Network Operations (CNO) — Computer Network Defense, Computer Network Exploitation, and Computer Network Attack.

So why a Fedora Red Team? My colleagues and I needed a place to work on offensive tooling, exploit curation, standards, and reference architectures — but we wanted to do it “the open source way.” A Fedora SIG gives us a community place to fail-fast these projects, a few of which I’ll mention here.

The Enterprise Linux Exploit Mapper (ELEM): as Mike Bursell wrote in his blog, many system administrators find themselves unable to patch. The CVE scoring system helps admins decide when to really push for patching, but many CVE descriptions contain language like “this vulnerability may allow an attacker to execute arbitrary code.” And there’s a reason for that — many vulnerabilities don’t have workable POCs. But what about the ones that do? ELEM makes it easy to map vulnerabilities on a local system to known exploits out in the wild. From a defensive perspective it creates a sort of super-criticality for admins so they can say to their management, “somebody can download this exploit and root our box right this minute.” A tool like this has good offensive value as well, and could save pentesters time doing these mappings manually.

The Fedora Cyber Test Lab (FCTL): I’ve written before about the Cyber-ITL and how important it is to the future of infosec. My only complaint is that their TTPs are not open source, and are not repeatable. So let’s fix that. This project will be an open source, automated, fully repeatable project for quantifying risk at a per-binary level. It will focus at first on RHEL, CentOS, and Fedora, but we’d love help from the community with adding other OS targets. I have a rudimentary first version ready to push to GitHub, which I’ll be blogging about in the coming days.

Penetration Testing Execution Standard (PTES): I’ve written before about how much I love PTES. In my mind, it’s a really important component of the art of pentesting. How can you tell a client that you’ve tested their security according to industry best practices without a yardstick by which to measure those practices? Without such a standard, pentesting relies too much on personal bona fides and flashy marketing. A standard like PTES can fix that. The only problem is that it hasn’t really been updated since 2014. Last summer, I rejiggered the wiki markup into ReStructured Text and put it on Read the Docs, which makes it easier to build community participation. The project is ready to be resurrected, and I hope that we’ll be able to work with the original team. But worst-case, we can fork it and go from there. This isn’t necessarily bad, and the PTES founders may have their reasons for wanting to let the project stay as it is. The Red Team SIG should know by early October which direction we’ll be taking.

I’m excited about this SIG, and will be haunting #fedora-security on Freenode IRC, as well as the security@lists.fedoraproject.org list. Every first Friday of the month we’ll be having meetings in IRC at 1400 UTC. Please join the conversation, or send us your pull requests for our GitHub projects.

Also, if you’re in Northern Virginia on 3 October 2017, we’ll be presenting ELEM at Defense in Depth. Register here and geek it up with us face-to-face!

Disclaimer: while I am talking about work stuff, this is my personal blog, and these views are my own and not my employer’s.

Make new KVM VMs in less than 10 seconds

In the course of my day, I tend to spin up lots of VMs on my laptop. KVM is my hypervisor of choice, and since it supports libvirt, there are lots of great tools to make this easier. virt-manager is a nice GUI that’s very helpful for beginners. virt-install is my CLI tool of choice. But if you want to use dnsmasq for guest name resolution, and dhcp against libvirt networking, it can be a little tedious to type out everything over and over. So I decided to make a tool to save me some time and typing: kvminstall.

Hat tip to Rich Lucente who shared with me a bash script that inspired me to write kvminstall.

Installation

To install, use Python PIP. If you haven’t used this before, it’s easy to install with yum.

# yum install python-pip
# pip install kvminstall
# kvminstall --help
usage: kvminstall [-h] [-c CLONE] [-i IMAGE] [-v VCPUS] [-r RAM] [-d DISK]
                  [-D DOMAIN] [-N NETWORK] [--type TYPE] [--variant VARIANT]
                  [-f CONFIGFILE] [--verbose]
                  name

positional arguments:
  name                  name of the new virtual machine

optional arguments:
  -h, --help            show this help message and exit
  -c CLONE, --clone CLONE
                        name of the source logical volume to be cloned
  -i IMAGE, --image IMAGE
                        image file to duplicate
  -v VCPUS, --vcpus VCPUS
                        number of virtual CPUs
  -r RAM, --ram RAM     amount of RAM in MB
  -d DISK, --disk DISK  disk size in GB
  -D DOMAIN, --domain DOMAIN
                        domainname for dhcp / dnsmasq
  -N NETWORK, --network NETWORK
                        libvirt network
  --type TYPE           os type, i.e., linux
  --variant VARIANT     os variant, i.e., rhel7
  -f CONFIGFILE, --configfile CONFIGFILE
                        specify an alternate config file,
                        default=~/.config/kvminstall/config.yaml
  --verbose             verbose output

Configuration

In your .config directory, kvminstall sets up a yaml file with defaults. You can specify any of these interactively, or if you want to minimize typing, you can set these defaults in ~/.config/kvminstall/config:

---
vcpus: 1
ram: 1024
disk: 10
domain: example.com
network: default
mac: 5c:e0:c5:c4:26
type: linux
variant: rhel7

The MAC address can be specified as up to 5 :-delimited fields. If you want to specify fewer, kvminstall will auto-complete with random, available values.

Usage

The current version 0.1.3 supports only image-based installs — either by snapshotting an LVM volume, or by copying an image file. I intend to add kickstart and iso support, but hey, release early, release often.

Image File

Most people will probably want to copy an image file. Let’s assume that you’ve built a base image, and its root volume lives in /var/lib/libvirt/images/rhel71base.img. (Next post will be on building base images.) To create a new VM, based on that image, called ‘testvm’:

# kvminstall -c /var/lib/libvirt/images/rhel71base.img testvm

You’re mostly I/O bound here, as your copying rhel71base.img -> testvm.img. Shortly after that’s finished, you’ve got a new VM with all of your host and guest networking configured.

# virsh list
 Id    Name                           State
----------------------------------------------------
 2     testvm                         running

# grep testvm /etc/hosts
192.168.122.27	testvm.example.com testvm
# ssh testvm
Last login: Thu Aug 27 13:30:25 2015 from 192.168.122.1
[root@testvm ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 5c:e0:c5:c4:26:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.27/24 brd 192.168.122.255 scope global dynamic eth0
       valid_lft 2141sec preferred_lft 2141sec
    inet6 fe80::5ee0:c5ff:fec4:267a/64 scope link 
       valid_lft forever preferred_lft forever
# nslookup testvm.example.com
Server:		192.168.122.1
Address:	192.168.122.1#53

Name:	testvm.example.com
Address: 192.168.122.27

The guest networking has been setup with virsh. An available IP and MAC address has been automatically picked based on your DHCP scope. (In the next version I’ll add support for specifying an IP address.)

# virsh net-dumpxml default
<network connections='1'>
  <name>default</name>
  <uuid>431ea266-8584-4e10-866a-fc1a3ad419b5</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:d0:5e:a3'/>
  <dns>
    <host ip='192.168.122.27'>
      <hostname>testvm.example.com</hostname>
    </host>
  </dns>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
      <host mac='5c:e0:c5:c4:26:7a' name='testvm.example.com' ip='192.168.122.27'/>
    </dhcp>
  </ip>
</network>

The dnsmasq service will be automatically restarted after /etc/hosts is updated. This way, so long as your resolv.conf is set up properly in your base image, DNS hostname resolution will work in your guest network.

LVM Volume

Now I use LVM volumes on my laptop, served up from an M2.SATA drive. This gives me better I/O since I’ve split out host and guest storage devices. It’s also much faster to snapshot a base image’s root volume. Using kvminstall with an LVM snapshot, you can get VM creation time down to seconds. My LVM volume group is called libvirt_lvm.

# lvs
  LV                 VG          Attr       LSize   Pool Origin     Data%  Meta%  Move Log Cpy%Sync Convert
  home               fedora      -wi-ao---- 500.00g                                                        
  root               fedora      -wi-ao---- 366.82g                                                        
  swap               fedora      -wi-ao----  64.00g                                                        
  rhel71base         libvirt_lvm owi-a-s---  10.00g                                                        
# time kvminstall -c /dev/libvirt_lvm/rhel71base testvm

real	0m2.217s
user	0m1.012s
sys	0m0.218s
[root@w550 ~]# ssh testvm
Warning: Permanently added the ECDSA host key for IP address '192.168.133.164' to the list of known hosts.
Last login: Sat Aug  8 21:02:29 2015 from 192.168.133.1
[root@testvm ~]# exit
# lvs
  LV                 VG          Attr       LSize   Pool Origin     Data%  Meta%  Move Log Cpy%Sync Convert
  home               fedora      -wi-ao---- 500.00g                                                        
  root               fedora      -wi-ao---- 366.82g                                                        
  swap               fedora      -wi-ao----  64.00g                                                               
  rhel71base         libvirt_lvm owi-a-s---  10.00g                                                        
  testvm             libvirt_lvm swi-aos---  10.00g      rhel71base 0.06                                   

Upcoming features

It would be nice if we could — just as quickly — remove the VMs, or even reset them back to their base images. In the next version, expect kvmuninstall and kvmreset commands.

I’d love feedback. Please feel free to comment here or open issues on the GitHub project page.

Stay tuned for my next article on building base images for easy cloning.