Lastline Labs

Lastline: It’s as easy as A-P-I

Posted by Dr. Giovanni Vigna LinkedIn Google+
4/18/16 9:19 AM

Lastline’s solutions analyze network traffic, programs, documents, and other artifacts to identify and block advanced malware in enterprise networks.

In order to be able to easily integrate its functionality in the security workflow of the enterprise, Lastline products provide access to their functionality through APIs.

At Lastline, we really believe that by using well-defined, well-documented APIs it is possible to combine best-of-breed solutions together to achieve effective overall security capabilities, such as the ability to detect and quarantine (or even re-image automatically) compromised hosts.

We don’t take APIs lightly. Every aspect of our solution is based on APIs. Even our user interface retrieves all the information displayed by the browser using APIs, and, different from any other solution out there, we make this explicit in the UI itself.

Take, for example, the figure below, which represents a portion of our user portal.

 

APItutorial1.png

In the top right corner there is a “</>” button. By clicking on that button, one can see the API request that was used to obtain the data used to generate the graph. You can see below a grab of what the request looked like.

APItutorial2.png

 

This is very handy in order to determine, just by looking at the UI, how the information could be retrieved. However, Lastline also provide extensive documentation on how to use its APIs in order to script complex tasks.

In the following, I will describe the step-by-step process to get up and running with Lastline’s APIs.

First of all, there are two main APIs: The Analyst API and the Lastline API (also called the Product API or PAPI).

The Analyst API focuses on submitting samples to Lastline’s analysis infrastructure (which can be hosted by Lastline or on-premise at the customer’s site) and retrieving the resulting reports. This blog post will focus on this API, while a second blog post will focus on how to use the Product API.

The documentation of both APIs is available from the customer portal, under the help menu, as shown in the screenshot below.

APItutorial3.png

 

But let’s get started.

The first step is, of course, to get a Lastline license and the corresponding API token, which is used for authentication.

The license looks like this: 2AAAD5A21DN0TBDFZZ66

The API token looks like this: IwoAGFa344c277Z2

Both credentials are generated at registration and are sent to the user in an email message.

We will use the above credentials in the rest of this blog (they are, obviously, fake).

To submit a file to the infrastructure we can simply use curl, a command-line tool available in most distributions.

Let’s choose a file from VirusTotal, for example:

https://www.virustotal.com/en/file/157162cf6ecfe9e0f584bf3d1249544437d6fc9810375cea549b0d8d310f6dae/analysis/

Usually, file submission is performed in two steps.

A first request is made to check if the file has already been analyzed. In this case, a hash of the file is sent (MD5 or SHA-1), in order to obtain a result.

The MD5 hash of the file is 5cafe3d4c826dc4652913a89d7bd444d.

The command-line request is:

curl https://analysis.lastline.com/analysis/submit/file -X POST -F "key=2AAAD5A21DN0TBDFZZ66" -F "api_token=IwoAGFa344c277Z2" -F "md5=5cafe3d4c826dc4652913a89d7bd444d"

Since the sample has already been analyzed, the result is a large json data structure. This is the  beginning of the returned data:

{"success": 1, "data": {"submission": "2016-03-09 21:04:16", "child_tasks": [], "reports": [{"relevance": 1.0, "report_versions": ["ll-int-win", "ll-win-timeline-based", "ioc:ll", "ioc:stix", "ioc:openioc", "ioc:openioc:tanium", "ll-win-timeline-thread-based"], "description": "Dynamic analysis on Microsoft Windows 7", "report_uuid":"2fbffe68406f50553d8d5400f3e3ef9c:575fea6c1138560cq22JObqvH_THQDOMJHix3lywJKFFpl8oRZudE96r5A4"},...

Note that since the sample was already known, the available reports are returned (in this case the dynamic analysis of the sample in both Windows 7 and Windows XP). One can then parse the json and extract interesting facts, such as the maliciousness score (field “score”), and the list of observed behaviors (field “malicious_activity”):

["Evasion: Switching processor mode from 32 to 64 bits (emulation escape)", "File: Modifying executable in root directory", "File: Modifying executable in suspicious location of application data directory", "Memory: Injecting code into browser process (chrome.exe)", "Memory: Injecting code into browser process (firefox.exe)", "Memory: Replacing the image of another process (detection evasion or privilege escalation)", "Memory: Writing to the memory of a non-child running process", "Network: Ability to download files from the Internet", "Network: Command&Control traffic observed", "Network: Connecting to server using hard-coded IP address", "Network: Hide network activity through code injection", "Packer: Loading an embedded PE image (potential unpacking)", "Packer: Overwriting Image Header (malicious packer)", "Search: Enumerates running processes", "Search: Retrieving the user account name", "Settings: Adding new trusted certificate to the system", "Signature: Identified trojan code", "Steal: Reading FTP client credentials", "Stealth: Creating hidden executable files"]

Let’s now take the same file, and modify one of its bytes to change its MD5 hash. As a result, we obtain a file that will behave identically but has a different hash, namely: 85b4abbf6b3cd97d048e369e66484573.

We now perform the same query for this new hash file:

curl https://analysis.lastline.com/analysis/submit/file -X POST -F "key=2AAAD5A21DN0TBDFZZ66" -F "api_token=IwoAGFa344c277Z2" -F "md5=85b4abbf6b3cd97d048e369e66484573”

This time, the json data we obtain is much shorter:

{"success": 0, "error_code": 101, "error": "No file found matching requested hash."}

The API is telling us that it needs the file to perform an analysis - this makes sense, since we just modified the file and we haven’t submitted it yet. Note that the system may return the same error despite having seen this file previously: this can happen if the analysis system decides to perform a new analysis (or parts of it), and the file is no-longer available in the analysis component.Therefore, we need to actually submit the file (which, in this case, is called “other.exe”):

curl https://analysis.lastline.com/analysis/submit/file -X POST -F "key=2AAAD5A21DN0TBDFZZ66" -F "api_token=IwoAGFa344c277Z2" -F "file=@other.exe”

What we get back is a task identifier:

{"success": 1, "data": {"task_uuid": "6738d71b044c4ff7ab07280181f28eac"}}

This tells us that the file has been submitted successfully and is now being processed.

We can check what tasks have been recently completed by calling the API endpoint get_completed, specifying that we want to know the tasks that have been completed after a specific time stamp (e.g., noon of March 11th, 2016, PST -- since the time must be in UTC, we need to add eight hours):

curl https://analysis.lastline.com/analysis/get_completed -X POST -F "key=2AAAD5A21DN0TBDFZZ66" -F "api_token=IwoAGFa344c277Z2" -F "after=2016-03-11 20:00:00"

The resulting json data tells us of what tasks have been completed since noon:

{"success": 1, "data": {"tasks": ["6738d71b044c4ff7ab07280181f28eac"], "after": "2016-03-11 20:00:00", "more_results_available": 0, "before": "2016-03-11 20:45:22"}}

As one can see, the task that we submitted has been completed.

Now, we are ready to request the result by calling the get_result endpoint:

curl https://analysis.lastline.com/analysis/get -X POST -F "key=2AAAD5A21DN0TBDFZZ66" -F "api_token=IwoAGFa344c277Z2" -F "uuid=6738d71b044c4ff7ab07280181f28eac"

The resulting json data contains all the reports that have been generated during the analysis of this samples.

{"success": 1, "data": {"submission": "2016-03-11 20:33:37", "child_tasks": [], "reports": [{"relevance": 0.55, "report_versions": ["ll-int-win", "ll-win-timeline-based", "ioc:ll", "ioc:stix", "ioc:openioc", "ioc:openioc:tanium", "ll-win-timeline-thread-based"], "description": "Dynamic analysis on Microsoft Windows XP", "report_uuid": "2fbffe68406f50553d8d5400f3e3ef9c:1cd12065e88c0bc1gZQo-WyzDu6SACX0Mcqnc6xY1PVupHmy5FR94WVo73c"}, {"relevance": 0.55, "report_versions": ["ll-int-win", "ll-win-timeline-based", "ioc:ll", "ioc:stix", "ioc:openioc", "ioc:openioc:tanium", "ll-win-timeline-thread-based"], "description": "Dynamic analysis on Microsoft Windows 7", "report_uuid": "2fbffe68406f50553d8d5400f3e3ef9c:737cb9037c28a797t7SgcBw9kSxNRrstBsnNrl9ql3PaUlmdmQJkQKjYX60"}], "task_uuid": "6738d71b044c4ff7ab07280181f28eac", "score": 100,...

Unsurprisingly, the sample is still considered malicious (score=100).

These few commands show how by using command-line tools, such as curl, one can directly interact with Lastline’s analysis infrastructure.

However, sometimes running commands from the command line and parsing json files is a bit cumbersome. Therefore, the Lastline user portal provides a Python module for easy access to the API endpoints.

It is sufficient to download the file called analysis_apiclient.py, put it in a directory called llapi_client, and create, in that directory an empty file called __init__.py.

Then it is possible to write very simple Pyhton program that will, for example, submit a sample and print its high-level behaviors:

import sys
from llapi_client import analysis_apiclient

analysis_client = analysis_apiclient.AnalysisClient(
   base_url="https://analysis.lastline.com",

  key="2AAAD5A21DN0TBDFZZ66",
  api_token="IwoAGFa344c277Z2")

filename_to_submit = sys.argv[1]

helper = analysis_apiclient.SubmissionHelper(analysis_client)
results = helper.submit_filenames_and_wait_for_completion(

  [filename_to_submit])
task_uuid = results[filename_to_submit].task_uuid
report = analysis_client.get_result(task_uuid)['data']
print report['malicious_activity']

The astute reader might have noticed that the above code has no actual error handling or attempt to be resilient with respect to user errors. It’s just a short example to showcase how easy it is to interact with the Analyst API.

In the next blog, we will combine the Analyst API with the Product API to collect more interesting information about a sample, and, instead of being just analysts, we will become hunters.




Topics: Lastline Enterprise, Integration, Lastline Labs, APIs

Subscribe to Email Updates