Introducing the Fedora Red Team

Last week my colleagues and I started a new Special Interest Group under the Fedora Project: the Fedora Red Team.

The Fedora Red Team’s goal is to become Red Hat’s upstream cybersecurity community. But what does that actually mean?

“Cyber” is a fascinating term with a colorful history. I’ve written about it before, but the punchline is that we owe its ubiquity to William Gibson’s Neuromancer and an obscure US Government paper from the late 90s, referred to as The Marsh Report.

The context of The Marsh Report seems to have stuck — cyber has come to refer to the high-stakes interplay of offensive and defensive infosec, often at nation-state scale. If you want to get ontological, you could say “cyber” includes the three sub-domains of Computer Network Operations (CNO) — Computer Network Defense, Computer Network Exploitation, and Computer Network Attack.

So why a Fedora Red Team? My colleagues and I needed a place to work on offensive tooling, exploit curation, standards, and reference architectures — but we wanted to do it “the open source way.” A Fedora SIG gives us a community place to fail-fast these projects, a few of which I’ll mention here.

The Enterprise Linux Exploit Mapper (ELEM): as Mike Bursell wrote in his blog, many system administrators find themselves unable to patch. The CVE scoring system helps admins decide when to really push for patching, but many CVE descriptions contain language like “this vulnerability may allow an attacker to execute arbitrary code.” And there’s a reason for that — many vulnerabilities don’t have workable POCs. But what about the ones that do? ELEM makes it easy to map vulnerabilities on a local system to known exploits out in the wild. From a defensive perspective it creates a sort of super-criticality for admins so they can say to their management, “somebody can download this exploit and root our box right this minute.” A tool like this has good offensive value as well, and could save pentesters time doing these mappings manually.

The Fedora Cyber Test Lab (FCTL): I’ve written before about the Cyber-ITL and how important it is to the future of infosec. My only complaint is that their TTPs are not open source, and are not repeatable. So let’s fix that. This project will be an open source, automated, fully repeatable project for quantifying risk at a per-binary level. It will focus at first on RHEL, CentOS, and Fedora, but we’d love help from the community with adding other OS targets. I have a rudimentary first version ready to push to GitHub, which I’ll be blogging about in the coming days.

Penetration Testing Execution Standard (PTES): I’ve written before about how much I love PTES. In my mind, it’s a really important component of the art of pentesting. How can you tell a client that you’ve tested their security according to industry best practices without a yardstick by which to measure those practices? Without such a standard, pentesting relies too much on personal bona fides and flashy marketing. A standard like PTES can fix that. The only problem is that it hasn’t really been updated since 2014. Last summer, I rejiggered the wiki markup into ReStructured Text and put it on Read the Docs, which makes it easier to build community participation. The project is ready to be resurrected, and I hope that we’ll be able to work with the original team. But worst-case, we can fork it and go from there. This isn’t necessarily bad, and the PTES founders may have their reasons for wanting to let the project stay as it is. The Red Team SIG should know by early October which direction we’ll be taking.

I’m excited about this SIG, and will be haunting #fedora-security on Freenode IRC, as well as the security@lists.fedoraproject.org list. Every first Friday of the month we’ll be having meetings in IRC at 1400 UTC. Please join the conversation, or send us your pull requests for our GitHub projects.

Also, if you’re in Northern Virginia on 3 October 2017, we’ll be presenting ELEM at Defense in Depth. Register here and geek it up with us face-to-face!

Disclaimer: while I am talking about work stuff, this is my personal blog, and these views are my own and not my employer’s.

Save your AWS budget with Python and boto

My team and I lean heavily on AWS services for prototypes, demos, and training. The challenge that we’ve encountered is that it’s easy to forget about the resources you’ve spun up. So I wrote a quickly little utility that shuts down unnecessary EC2 instances at night.

The Python library, boto, provides an AWS SDK. It’s very easy to use, and many good tutorials exist. Instructions can be found in the README, but here’s a quick overview.

First we import the boto and yaml libraries. (We’re using YAML for our config file markup. ) Then we read in that config file.

import boto.ec2
import yaml

config_file = '/etc/nightly_shutdown.yml'
with open(config_file) as f:
    config = yaml.safe_load(f)

In that config file, we’ve got our region, access and secret keys, and a white list of instance IDs we’d like to opt-out of the nightly shutdown. This last bit is important if you have instances doing long-running jobs like repo syncing, for example.

---
region: us-east-1
access_key: eggseggseggseggs
secret_key: spamspamspamspam
whitelist:
  - i-abcdefgh
  - i-ijklmnop

Now we connect to the AWS API and get a list of reservations. This itself is interesting, as it gives us a little insight into the guts of EC2. As I understand it, a reservation must exist before an instance can be launched.

conn = boto.ec2.connect_to_region(config['region'],
                                  aws_access_key_id=config['access_key'],
                                  aws_secret_access_key=config['secret_key'])

reservations = conn.get_all_reservations()

Now it’s simply a matter of iterating over those reservations, getting the instance IDs, and filtering out the white-listed IDs.

running_instances = []
for r in reservations:
    for i in r.instances:
        if i.state == "running":
            if i.id not in config['whitelist']:
                running_instances.append(i.id)

Finally, we make the API call to stop the instances. Before doing so, we check to be sure there are any running, as this call will throw an exception if the instance ID list is empty.

if len(running_instances) > 0:
    conn.stop_instances(instance_ids=running_instances)

Now you just have to add this to your daily cronjobs and you’ll save a little budget.

Cyber-ITL Round up

I was talking to a friend about the Cyber-ITL. His reaction was, “Wat?” So in case you missed it, an important thing is happening. EDIT: the BlackHat video was DMCAed. Here’s the Def Con version instead, which is better anyway.

Mudge and his wife, Sarah, presented this at BlackHat and Def Con this year.

If you watch only one video in November, make it this one. This is extremely important, and plays a big part in things to come.

Related:

The Cyber-ITL site itself is a little sparse; Mudge has been slowed down a bit by health problems. But there are a few good articles to read:

The PTES pentesting standard is awesome and you should read it

If you’re into pentesting or red teaming, sooner or later you’ll encounter some standardized methodologies.

The National Institute of Standards and Technologies (NIST) has one called the “Technical Guide to Information Security Testing and Assessment,” or SP800-115. I’m a big fan of NIST, and this is a good place to start, especially if you care about FISMA risk management frameworks. But it’s pretty high-level, and will probably leave you wanting more.

With a little more Googling, you’ll then find pentest-standard.org. The page has a dated MediaWiki interface. It hasn’t been updated in almost a year. But those things don’t matter, this site is made of open source awesomeness.

The meat of the site lives in the PTES Technical Guidelines. It’s fairly extensive, and if you’re already somewhat familiar with information security, it can go a long way to teaching you about penetration testing.

To give you an idea of the scope of this methodology, take a look at the FreeMind map that they posted, converted here to PNG for your viewing ease.

penetration_testing_execution_standard

Go ahead and click on it, you’ll need to load the whole thing then zoom. It’s enormous.

Every one of these entries in the mindmap are backed up by some direction in the Technical Guidelines. Granted, PTES doesn’t hold your hand in all places, but for the devoted student of pentesting, this is invaluable stuff.

Now, to be fair, PTES is not the only game in town. There are other methodologies worth mentioning; I’ll write more about the later, but here’s an overview.

OWASP is another open source pentesting framework, but it’s focused at the web application layer. 18F, the folks behind cloud.gov and other cool stuff, requires the use of an OWASP automated scanner called ZAP as part of the ATO process.

ISSAF is another cool methodology, but it’s even harder to navigate than PTES. You can download the rar archive, or navigate the individual .doc files. At some point I hope to map PTES and ISSAF steps to one another to identify gaps in the former and contribute back to the project.

As much as I like it, PTES could really use a little TLC. There are incomplete sections. And a more modern interface would help, possibly even a migration to a GitHub Pages model, which would make community contribution easier. A D3 directed graph (example) would make for a nice, interactive mindmap.

But despite its shortcomings, I’d say it’s still the best open source pentesting methodology out there. Go check it out.

Origins of the term “cyber”

I used to rail against the term “cyber.” As one who grew up during the transitional period between ARPANET and the Internet, “cyber” took on ridiculous meaning. One learned quickly to avoid anyone whose avatar — or blinking cursor back then — asked, “wanna cyber?” Visions of Max Headroom doing unspeakable things were, for me, mentally far too near.

Earlier this year, Fred Kaplan published an excellent book called Dark Territory: The Secret History of Cyber War. It was this book that completely changed my mind.

I’m now happy to admit that the term, “cyber,” is actually really cool.

First, let’s look at the origin of the word. Wikipedia tells us that the cyber prefix comes from the word, “cybernetic.” This is cool in and of itself, because the word “cybernetics” dates all the way back to Plato. Digging deeper, we see that the etymology of cybernetics comes from a word familiar to the DevOps world: κυβερνήτης, or transliterated, kubernetesEtymonline confirms it:

cybernetics (n.)
coined 1948 by U.S. mathematician Norbert Wiener (1894-1964) from Greek kybernetes “steersman” (metaphorically “guide, governor”)

This is already pretty great, but the rabbit hole goes deeper. Back to Kaplan (pp. 44-45):

What to call these “other” threats? One word was floating around in stories about hackings of one sort or another: “cyber.” …the term stemmed from William Gibson’s 1984 science-fiction novel, Neuromancer, a wild and eerily prescient tale of murder and mayhem in the virtual world of “cyberspace.”

Kaplan goes on to describe how a Justice Department attorney named Michael Vatis had just read Neuromancer, and suggested they use the term during the development of the Marsh Report. This pleases me greatly, as Neuromancer is probably one of my top three favorite books of all time.

So there you have it — Plato, William Gibson, and Kubernetes all tied up in one awesome word. Now when you hear terms like cyberwar, cybercrime, cyber-anything you can rest assured that the term is not lame, but indeed, quite kick-ass.

Python one-liner: converting JSON to YAML

I’ve been playing with the Titan graph database lately; it’s hella cool, super powerful, and has a great ecosystem. One tool in the Titan toolbox is a REST interface called Rexster.

You can check to see that it’s up and what it’s serving up by curl-ing one of its endpoints.

# curl localhost:8182/graphs/graph
{"version":"2.5.0","name":"graph","graph":"titangraph[cassandrathrift:[127.0.0.1]]","features":{"isWrapper":false,"supportsVertexProperties":true,"supportsMapProperty":true,"supportsUniformListProperty":true,"supportsIndices":false,"ignoresSuppliedIds":true,"supportsFloatProperty":true,"supportsPrimitiveArrayProperty":true,"supportsEdgeIndex":false,"supportsKeyIndices":true,"supportsDoubleProperty":true,"isPersistent":true,"supportsVertexIteration":true,"supportsEdgeProperties":true,"supportsSelfLoops":true,"supportsDuplicateEdges":true,"supportsSerializableObjectProperty":true,"supportsEdgeIteration":true,"supportsVertexIndex":false,"supportsIntegerProperty":true,"supportsBooleanProperty":true,"supportsMixedListProperty":true,"supportsEdgeRetrieval":true,"supportsTransactions":true,"supportsThreadedTransactions":true,"supportsStringProperty":true,"supportsVertexKeyIndex":false,"supportsEdgeKeyIndex":false,"supportsLongProperty":true},"readOnly":false,"type":"com.thinkaurelius.titan.graphdb.database.StandardTitanGraph","queryTime":0.213622,"upTime":"0[d]:00[h]:28[m]:25[s]","extensions":[{"op":"GET","namespace":"tp","name":"gremlin","description":"evaluate an ad-hoc Gremlin script for a graph.","href":"http://localhost:8182/graphs/graph/tp/gremlin","title":"tp:gremlin","parameters":[{"name":"rexster.showTypes","description":"displays the properties of the elements with their native data type (default is false)"},{"name":"language","description":"the gremlin language flavor to use (default is groovy)"},{"name":"params","description":"a map of parameters to bind to the script engine"},{"name":"load","description":"a list of 'stored procedures' to execute prior to the 'script' (if 'script' is not specified then the last script in this argument will return the values"},{"name":"returnTotal","description":"when set to true, the full result set will be iterated and the results returned (default is false)"},{"name":"rexster.returnKeys","description":"an array of element property keys to return (default is to return all element properties)"},{"name":"rexster.offset.start","description":"start index for a paged set of data to be returned"},{"name":"rexster.offset.end","description":"end index for a paged set of data to be returned"},{"name":"script","description":"the Gremlin script to be evaluated"}]},{"op":"POST","namespace":"tp","name":"gremlin","description":"evaluate an ad-hoc Gremlin script for a graph.","href":"http://localhost:8182/graphs/graph/tp/gremlin","title":"tp:gremlin","parameters":[{"name":"rexster.showTypes","description":"displays the properties of the elements with their native data type (default is false)"},{"name":"language","description":"the gremlin language flavor to use (default is groovy)"},{"name":"params","description":"a map of parameters to bind to the script engine"},{"name":"load","description":"a list of 'stored procedures' to execute prior to the 'script' (if 'script' is not specified then the last script in this argument will return the values"},{"name":"returnTotal","description":"when set to true, the full result set will be iterated and the results returned (default is false)"},{"name":"rexster.returnKeys","description":"an array of element property keys to return (default is to return all element properties)"},{"name":"rexster.offset.start","description":"start index for a paged set of data to be returned"},{"name":"rexster.offset.end","description":"end index for a paged set of data to be returned"},{"name":"script","description":"the Gremlin script to be evaluated"}]}

Ugly. Python to the rescue.

#!/usr/bin/env python

import simplejson
import sys
import yaml

print yaml.dump(simplejson.loads(str(sys.stdin.read())), default_flow_style=False)

Basically a one-liner.

# curl localhost:32791/graphs/graph | python json2yaml.py 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3581    0  3581    0     0   552k      0 --:--:-- --:--:-- --:--:--  582k
extensions:
- description: evaluate an ad-hoc Gremlin script for a graph.
  href: http://localhost:8182/graphs/graph/tp/gremlin
  name: gremlin
  namespace: tp
  op: GET
  parameters:
  - description: displays the properties of the elements with their native data type
      (default is false)
    name: rexster.showTypes
  - description: the gremlin language flavor to use (default is groovy)
    name: language
  - description: a map of parameters to bind to the script engine
    name: params
  - description: a list of 'stored procedures' to execute prior to the 'script' (if
      'script' is not specified then the last script in this argument will return
      the values
    name: load
  - description: when set to true, the full result set will be iterated and the results
      returned (default is false)
    name: returnTotal
  - description: an array of element property keys to return (default is to return
      all element properties)
    name: rexster.returnKeys
  - description: start index for a paged set of data to be returned
    name: rexster.offset.start
  - description: end index for a paged set of data to be returned
    name: rexster.offset.end
  - description: the Gremlin script to be evaluated
    name: script
  title: tp:gremlin
- description: evaluate an ad-hoc Gremlin script for a graph.
  href: http://localhost:8182/graphs/graph/tp/gremlin
  name: gremlin
  namespace: tp
  op: POST
  parameters:
  - description: displays the properties of the elements with their native data type
      (default is false)
    name: rexster.showTypes
  - description: the gremlin language flavor to use (default is groovy)
    name: language
  - description: a map of parameters to bind to the script engine
    name: params
  - description: a list of 'stored procedures' to execute prior to the 'script' (if
      'script' is not specified then the last script in this argument will return
      the values
    name: load
  - description: when set to true, the full result set will be iterated and the results
      returned (default is false)
    name: returnTotal
  - description: an array of element property keys to return (default is to return
      all element properties)
    name: rexster.returnKeys
  - description: start index for a paged set of data to be returned
    name: rexster.offset.start
  - description: end index for a paged set of data to be returned
    name: rexster.offset.end
  - description: the Gremlin script to be evaluated
    name: script
  title: tp:gremlin
features:
  ignoresSuppliedIds: true
  isPersistent: true
  isWrapper: false
  supportsBooleanProperty: true
  supportsDoubleProperty: true
  supportsDuplicateEdges: true
  supportsEdgeIndex: false
  supportsEdgeIteration: true
  supportsEdgeKeyIndex: false
  supportsEdgeProperties: true
  supportsEdgeRetrieval: true
  supportsFloatProperty: true
  supportsIndices: false
  supportsIntegerProperty: true
  supportsKeyIndices: true
  supportsLongProperty: true
  supportsMapProperty: true
  supportsMixedListProperty: true
  supportsPrimitiveArrayProperty: true
  supportsSelfLoops: true
  supportsSerializableObjectProperty: true
  supportsStringProperty: true
  supportsThreadedTransactions: true
  supportsTransactions: true
  supportsUniformListProperty: true
  supportsVertexIndex: false
  supportsVertexIteration: true
  supportsVertexKeyIndex: false
  supportsVertexProperties: true
graph: titangraph[cassandrathrift:[127.0.0.1]]
name: graph
queryTime: 0.31277
readOnly: false
type: com.thinkaurelius.titan.graphdb.database.StandardTitanGraph
upTime: 0[d]:00[h]:31[m]:27[s]
version: 2.5.0

I love Python. YAML ain’t bad, either.