Cyber-ITL Round up

I was talking to a friend about the Cyber-ITL. His reaction was, “Wat?” So in case you missed it, an important thing is happening. EDIT: the BlackHat video was DMCAed. Here’s the Def Con version instead, which is better anyway.

Mudge and his wife, Sarah, presented this at BlackHat and Def Con this year.

If you watch only one video in November, make it this one. This is extremely important, and plays a big part in things to come.

Related:

The Cyber-ITL site itself is a little sparse; Mudge has been slowed down a bit by health problems. But there are a few good articles to read:

The PTES pentesting standard is awesome and you should read it

If you’re into pentesting or red teaming, sooner or later you’ll encounter some standardized methodologies.

The National Institute of Standards and Technologies (NIST) has one called the “Technical Guide to Information Security Testing and Assessment,” or SP800-115. I’m a big fan of NIST, and this is a good place to start, especially if you care about FISMA risk management frameworks. But it’s pretty high-level, and will probably leave you wanting more.

With a little more Googling, you’ll then find pentest-standard.org. The page has a dated MediaWiki interface. It hasn’t been updated in almost a year. But those things don’t matter, this site is made of open source awesomeness.

The meat of the site lives in the PTES Technical Guidelines. It’s fairly extensive, and if you’re already somewhat familiar with information security, it can go a long way to teaching you about penetration testing.

To give you an idea of the scope of this methodology, take a look at the FreeMind map that they posted, converted here to PNG for your viewing ease.

penetration_testing_execution_standard

Go ahead and click on it, you’ll need to load the whole thing then zoom. It’s enormous.

Every one of these entries in the mindmap are backed up by some direction in the Technical Guidelines. Granted, PTES doesn’t hold your hand in all places, but for the devoted student of pentesting, this is invaluable stuff.

Now, to be fair, PTES is not the only game in town. There are other methodologies worth mentioning; I’ll write more about the later, but here’s an overview.

OWASP is another open source pentesting framework, but it’s focused at the web application layer. 18F, the folks behind cloud.gov and other cool stuff, requires the use of an OWASP automated scanner called ZAP as part of the ATO process.

ISSAF is another cool methodology, but it’s even harder to navigate than PTES. You can download the rar archive, or navigate the individual .doc files. At some point I hope to map PTES and ISSAF steps to one another to identify gaps in the former and contribute back to the project.

As much as I like it, PTES could really use a little TLC. There are incomplete sections. And a more modern interface would help, possibly even a migration to a GitHub Pages model, which would make community contribution easier. A D3 directed graph (example) would make for a nice, interactive mindmap.

But despite its shortcomings, I’d say it’s still the best open source pentesting methodology out there. Go check it out.

Origins of the term “cyber”

I used to rail against the term “cyber.” As one who grew up during the transitional period between ARPANET and the Internet, “cyber” took on ridiculous meaning. One learned quickly to avoid anyone whose avatar — or blinking cursor back then — asked, “wanna cyber?” Visions of Max Headroom doing unspeakable things were, for me, mentally far too near.

Earlier this year, Fred Kaplan published an excellent book called Dark Territory: The Secret History of Cyber War. It was this book that completely changed my mind.

I’m now happy to admit that the term, “cyber,” is actually really cool.

First, let’s look at the origin of the word. Wikipedia tells us that the cyber prefix comes from the word, “cybernetic.” This is cool in and of itself, because the word “cybernetics” dates all the way back to Plato. Digging deeper, we see that the etymology of cybernetics comes from a word familiar to the DevOps world: κυβερνήτης, or transliterated, kubernetesEtymonline confirms it:

cybernetics (n.)
coined 1948 by U.S. mathematician Norbert Wiener (1894-1964) from Greek kybernetes “steersman” (metaphorically “guide, governor”)

This is already pretty great, but the rabbit hole goes deeper. Back to Kaplan (pp. 44-45):

What to call these “other” threats? One word was floating around in stories about hackings of one sort or another: “cyber.” …the term stemmed from William Gibson’s 1984 science-fiction novel, Neuromancer, a wild and eerily prescient tale of murder and mayhem in the virtual world of “cyberspace.”

Kaplan goes on to describe how a Justice Department attorney named Michael Vatis had just read Neuromancer, and suggested they use the term during the development of the Marsh Report. This pleases me greatly, as Neuromancer is probably one of my top three favorite books of all time.

So there you have it — Plato, William Gibson, and Kubernetes all tied up in one awesome word. Now when you hear terms like cyberwar, cybercrime, cyber-anything you can rest assured that the term is not lame, but indeed, quite kick-ass.

Python one-liner: converting JSON to YAML

I’ve been playing with the Titan graph database lately; it’s hella cool, super powerful, and has a great ecosystem. One tool in the Titan toolbox is a REST interface called Rexster.

You can check to see that it’s up and what it’s serving up by curl-ing one of its endpoints.

# curl localhost:8182/graphs/graph
{"version":"2.5.0","name":"graph","graph":"titangraph[cassandrathrift:[127.0.0.1]]","features":{"isWrapper":false,"supportsVertexProperties":true,"supportsMapProperty":true,"supportsUniformListProperty":true,"supportsIndices":false,"ignoresSuppliedIds":true,"supportsFloatProperty":true,"supportsPrimitiveArrayProperty":true,"supportsEdgeIndex":false,"supportsKeyIndices":true,"supportsDoubleProperty":true,"isPersistent":true,"supportsVertexIteration":true,"supportsEdgeProperties":true,"supportsSelfLoops":true,"supportsDuplicateEdges":true,"supportsSerializableObjectProperty":true,"supportsEdgeIteration":true,"supportsVertexIndex":false,"supportsIntegerProperty":true,"supportsBooleanProperty":true,"supportsMixedListProperty":true,"supportsEdgeRetrieval":true,"supportsTransactions":true,"supportsThreadedTransactions":true,"supportsStringProperty":true,"supportsVertexKeyIndex":false,"supportsEdgeKeyIndex":false,"supportsLongProperty":true},"readOnly":false,"type":"com.thinkaurelius.titan.graphdb.database.StandardTitanGraph","queryTime":0.213622,"upTime":"0[d]:00[h]:28[m]:25[s]","extensions":[{"op":"GET","namespace":"tp","name":"gremlin","description":"evaluate an ad-hoc Gremlin script for a graph.","href":"http://localhost:8182/graphs/graph/tp/gremlin","title":"tp:gremlin","parameters":[{"name":"rexster.showTypes","description":"displays the properties of the elements with their native data type (default is false)"},{"name":"language","description":"the gremlin language flavor to use (default is groovy)"},{"name":"params","description":"a map of parameters to bind to the script engine"},{"name":"load","description":"a list of 'stored procedures' to execute prior to the 'script' (if 'script' is not specified then the last script in this argument will return the values"},{"name":"returnTotal","description":"when set to true, the full result set will be iterated and the results returned (default is false)"},{"name":"rexster.returnKeys","description":"an array of element property keys to return (default is to return all element properties)"},{"name":"rexster.offset.start","description":"start index for a paged set of data to be returned"},{"name":"rexster.offset.end","description":"end index for a paged set of data to be returned"},{"name":"script","description":"the Gremlin script to be evaluated"}]},{"op":"POST","namespace":"tp","name":"gremlin","description":"evaluate an ad-hoc Gremlin script for a graph.","href":"http://localhost:8182/graphs/graph/tp/gremlin","title":"tp:gremlin","parameters":[{"name":"rexster.showTypes","description":"displays the properties of the elements with their native data type (default is false)"},{"name":"language","description":"the gremlin language flavor to use (default is groovy)"},{"name":"params","description":"a map of parameters to bind to the script engine"},{"name":"load","description":"a list of 'stored procedures' to execute prior to the 'script' (if 'script' is not specified then the last script in this argument will return the values"},{"name":"returnTotal","description":"when set to true, the full result set will be iterated and the results returned (default is false)"},{"name":"rexster.returnKeys","description":"an array of element property keys to return (default is to return all element properties)"},{"name":"rexster.offset.start","description":"start index for a paged set of data to be returned"},{"name":"rexster.offset.end","description":"end index for a paged set of data to be returned"},{"name":"script","description":"the Gremlin script to be evaluated"}]}

Ugly. Python to the rescue.

#!/usr/bin/env python

import simplejson
import sys
import yaml

print yaml.dump(simplejson.loads(str(sys.stdin.read())), default_flow_style=False)

Basically a one-liner.

# curl localhost:32791/graphs/graph | python json2yaml.py 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3581    0  3581    0     0   552k      0 --:--:-- --:--:-- --:--:--  582k
extensions:
- description: evaluate an ad-hoc Gremlin script for a graph.
  href: http://localhost:8182/graphs/graph/tp/gremlin
  name: gremlin
  namespace: tp
  op: GET
  parameters:
  - description: displays the properties of the elements with their native data type
      (default is false)
    name: rexster.showTypes
  - description: the gremlin language flavor to use (default is groovy)
    name: language
  - description: a map of parameters to bind to the script engine
    name: params
  - description: a list of 'stored procedures' to execute prior to the 'script' (if
      'script' is not specified then the last script in this argument will return
      the values
    name: load
  - description: when set to true, the full result set will be iterated and the results
      returned (default is false)
    name: returnTotal
  - description: an array of element property keys to return (default is to return
      all element properties)
    name: rexster.returnKeys
  - description: start index for a paged set of data to be returned
    name: rexster.offset.start
  - description: end index for a paged set of data to be returned
    name: rexster.offset.end
  - description: the Gremlin script to be evaluated
    name: script
  title: tp:gremlin
- description: evaluate an ad-hoc Gremlin script for a graph.
  href: http://localhost:8182/graphs/graph/tp/gremlin
  name: gremlin
  namespace: tp
  op: POST
  parameters:
  - description: displays the properties of the elements with their native data type
      (default is false)
    name: rexster.showTypes
  - description: the gremlin language flavor to use (default is groovy)
    name: language
  - description: a map of parameters to bind to the script engine
    name: params
  - description: a list of 'stored procedures' to execute prior to the 'script' (if
      'script' is not specified then the last script in this argument will return
      the values
    name: load
  - description: when set to true, the full result set will be iterated and the results
      returned (default is false)
    name: returnTotal
  - description: an array of element property keys to return (default is to return
      all element properties)
    name: rexster.returnKeys
  - description: start index for a paged set of data to be returned
    name: rexster.offset.start
  - description: end index for a paged set of data to be returned
    name: rexster.offset.end
  - description: the Gremlin script to be evaluated
    name: script
  title: tp:gremlin
features:
  ignoresSuppliedIds: true
  isPersistent: true
  isWrapper: false
  supportsBooleanProperty: true
  supportsDoubleProperty: true
  supportsDuplicateEdges: true
  supportsEdgeIndex: false
  supportsEdgeIteration: true
  supportsEdgeKeyIndex: false
  supportsEdgeProperties: true
  supportsEdgeRetrieval: true
  supportsFloatProperty: true
  supportsIndices: false
  supportsIntegerProperty: true
  supportsKeyIndices: true
  supportsLongProperty: true
  supportsMapProperty: true
  supportsMixedListProperty: true
  supportsPrimitiveArrayProperty: true
  supportsSelfLoops: true
  supportsSerializableObjectProperty: true
  supportsStringProperty: true
  supportsThreadedTransactions: true
  supportsTransactions: true
  supportsUniformListProperty: true
  supportsVertexIndex: false
  supportsVertexIteration: true
  supportsVertexKeyIndex: false
  supportsVertexProperties: true
graph: titangraph[cassandrathrift:[127.0.0.1]]
name: graph
queryTime: 0.31277
readOnly: false
type: com.thinkaurelius.titan.graphdb.database.StandardTitanGraph
upTime: 0[d]:00[h]:31[m]:27[s]
version: 2.5.0

I love Python. YAML ain’t bad, either.

GitHub two factor authentication with IntelliJ

I’m a big fan of the IntelliJ products and derivatives, particularly Pycharm and Android Studio.

I also use two factor authentication (2fa) on every site that supports it. GitHub, no stranger to awesomeness, supports 2fa like a boss!

The easiest way to make your IntelliJ IDE jive with your 2fa-enabled GitHub account is to use personal API tokens. You have to be careful with these, because they’re a form of single-factor authentication, but since they’re long, random, and typically used for one purpose (i.e., you’re IDE), I think their overall impact to your account’s security is acceptable.

After you’ve created your personal API token (I used the default settings), open your settings dialog in your IntelliJ IDE. pycharm_github_settings

For “Auth Type” pick “Token.” Insert your token into the field, click “Test” to see if it worked, and you’re good to go!

Make new KVM VMs in less than 10 seconds

In the course of my day, I tend to spin up lots of VMs on my laptop. KVM is my hypervisor of choice, and since it supports libvirt, there are lots of great tools to make this easier. virt-manager is a nice GUI that’s very helpful for beginners. virt-install is my CLI tool of choice. But if you want to use dnsmasq for guest name resolution, and dhcp against libvirt networking, it can be a little tedious to type out everything over and over. So I decided to make a tool to save me some time and typing: kvminstall.

Hat tip to Rich Lucente who shared with me a bash script that inspired me to write kvminstall.

Installation

To install, use Python PIP. If you haven’t used this before, it’s easy to install with yum.

# yum install python-pip
# pip install kvminstall
# kvminstall --help
usage: kvminstall [-h] [-c CLONE] [-i IMAGE] [-v VCPUS] [-r RAM] [-d DISK]
                  [-D DOMAIN] [-N NETWORK] [--type TYPE] [--variant VARIANT]
                  [-f CONFIGFILE] [--verbose]
                  name

positional arguments:
  name                  name of the new virtual machine

optional arguments:
  -h, --help            show this help message and exit
  -c CLONE, --clone CLONE
                        name of the source logical volume to be cloned
  -i IMAGE, --image IMAGE
                        image file to duplicate
  -v VCPUS, --vcpus VCPUS
                        number of virtual CPUs
  -r RAM, --ram RAM     amount of RAM in MB
  -d DISK, --disk DISK  disk size in GB
  -D DOMAIN, --domain DOMAIN
                        domainname for dhcp / dnsmasq
  -N NETWORK, --network NETWORK
                        libvirt network
  --type TYPE           os type, i.e., linux
  --variant VARIANT     os variant, i.e., rhel7
  -f CONFIGFILE, --configfile CONFIGFILE
                        specify an alternate config file,
                        default=~/.config/kvminstall/config.yaml
  --verbose             verbose output

Configuration

In your .config directory, kvminstall sets up a yaml file with defaults. You can specify any of these interactively, or if you want to minimize typing, you can set these defaults in ~/.config/kvminstall/config:

---
vcpus: 1
ram: 1024
disk: 10
domain: example.com
network: default
mac: 5c:e0:c5:c4:26
type: linux
variant: rhel7

The MAC address can be specified as up to 5 :-delimited fields. If you want to specify fewer, kvminstall will auto-complete with random, available values.

Usage

The current version 0.1.3 supports only image-based installs — either by snapshotting an LVM volume, or by copying an image file. I intend to add kickstart and iso support, but hey, release early, release often.

Image File

Most people will probably want to copy an image file. Let’s assume that you’ve built a base image, and its root volume lives in /var/lib/libvirt/images/rhel71base.img. (Next post will be on building base images.) To create a new VM, based on that image, called ‘testvm’:

# kvminstall -c /var/lib/libvirt/images/rhel71base.img testvm

You’re mostly I/O bound here, as your copying rhel71base.img -> testvm.img. Shortly after that’s finished, you’ve got a new VM with all of your host and guest networking configured.

# virsh list
 Id    Name                           State
----------------------------------------------------
 2     testvm                         running

# grep testvm /etc/hosts
192.168.122.27	testvm.example.com testvm
# ssh testvm
Last login: Thu Aug 27 13:30:25 2015 from 192.168.122.1
[root@testvm ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 5c:e0:c5:c4:26:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.27/24 brd 192.168.122.255 scope global dynamic eth0
       valid_lft 2141sec preferred_lft 2141sec
    inet6 fe80::5ee0:c5ff:fec4:267a/64 scope link 
       valid_lft forever preferred_lft forever
# nslookup testvm.example.com
Server:		192.168.122.1
Address:	192.168.122.1#53

Name:	testvm.example.com
Address: 192.168.122.27

The guest networking has been setup with virsh. An available IP and MAC address has been automatically picked based on your DHCP scope. (In the next version I’ll add support for specifying an IP address.)

# virsh net-dumpxml default
<network connections='1'>
  <name>default</name>
  <uuid>431ea266-8584-4e10-866a-fc1a3ad419b5</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:d0:5e:a3'/>
  <dns>
    <host ip='192.168.122.27'>
      <hostname>testvm.example.com</hostname>
    </host>
  </dns>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
      <host mac='5c:e0:c5:c4:26:7a' name='testvm.example.com' ip='192.168.122.27'/>
    </dhcp>
  </ip>
</network>

The dnsmasq service will be automatically restarted after /etc/hosts is updated. This way, so long as your resolv.conf is set up properly in your base image, DNS hostname resolution will work in your guest network.

LVM Volume

Now I use LVM volumes on my laptop, served up from an M2.SATA drive. This gives me better I/O since I’ve split out host and guest storage devices. It’s also much faster to snapshot a base image’s root volume. Using kvminstall with an LVM snapshot, you can get VM creation time down to seconds. My LVM volume group is called libvirt_lvm.

# lvs
  LV                 VG          Attr       LSize   Pool Origin     Data%  Meta%  Move Log Cpy%Sync Convert
  home               fedora      -wi-ao---- 500.00g                                                        
  root               fedora      -wi-ao---- 366.82g                                                        
  swap               fedora      -wi-ao----  64.00g                                                        
  rhel71base         libvirt_lvm owi-a-s---  10.00g                                                        
# time kvminstall -c /dev/libvirt_lvm/rhel71base testvm

real	0m2.217s
user	0m1.012s
sys	0m0.218s
[root@w550 ~]# ssh testvm
Warning: Permanently added the ECDSA host key for IP address '192.168.133.164' to the list of known hosts.
Last login: Sat Aug  8 21:02:29 2015 from 192.168.133.1
[root@testvm ~]# exit
# lvs
  LV                 VG          Attr       LSize   Pool Origin     Data%  Meta%  Move Log Cpy%Sync Convert
  home               fedora      -wi-ao---- 500.00g                                                        
  root               fedora      -wi-ao---- 366.82g                                                        
  swap               fedora      -wi-ao----  64.00g                                                               
  rhel71base         libvirt_lvm owi-a-s---  10.00g                                                        
  testvm             libvirt_lvm swi-aos---  10.00g      rhel71base 0.06                                   

Upcoming features

It would be nice if we could — just as quickly — remove the VMs, or even reset them back to their base images. In the next version, expect kvmuninstall and kvmreset commands.

I’d love feedback. Please feel free to comment here or open issues on the GitHub project page.

Stay tuned for my next article on building base images for easy cloning.