Tuesday, May 30, 2017

A quick note on imaging newer Android devices

Actually a quick note

All blog posts to date
I was on the phone with a good friend of mine earlier this week.  He called me long-winded.  According to my wife, my family, my friends, and my coworkers, the statement was accurate.  So I'll make this one not so long-winded.

In a previous post, I demonstrated how to make a physical image of a device.  So let's say you have a rooted newer device, like Android 7.0 or newer, and you follow that guide and image /dev/block/mmcblk0.  You open the image in FTK Imager or any other viewer of choice, and it all looks good until you get to the userdata partition.  You get the dreaded "cannot read filesystem" or "unknown file system" or other such error.  You get ticked off because you just spent an hour plus imaging the device, and now it looks like the most important partition by a long shot imaged wrong.  So you go back and do it again and receive the same results.  Now you've wasted two plus hours.  I'm here to save you from wasting further hours.

File by file encryption
By default, many newer builds of Android include file-based encryption on the userdata partition.  The long and short of it is the entire partition is not encrypted, but each file is.  So if you capture the partition with no attempt to decrypt or otherwise circumvent the encryption, you will not be able to view the data.

Now users can set up more complicated encryption.  If that's the case, I don't think the method below is going to work.  I'm talking about devices where the user just uses a simple pin or fingerprint lock, not a fully-encrypted device.
So when you image /dev/block/mmcblk0, you image the entire internal storage, beginning to end.  The problem here is imaging that entire internal storage grabs an encrypted version of userdata.  So we need to image a decrypted version.

Check out my previous post on identifying your userdata partition.  In the post, I explain how to use the "mount" command to find the block mounted at /data.  That block is your userdata, and if you image that, you get just the userdata partition.

As it works out, that same method can bypass the Android 7.0 file based encryption (again, so long as the device is not fully encrypted).

So if you have such a device, adb shell into it and type the following command:

You will see a list of all mounted partitions.  One of them might look something like this (mind the edits for making it a bit generic) ...

/dev/block/platform/something/dm-0 /data ext4 rw,bunch of other mount commands

Point is, find the one mounted at /data.  Image just that one.  See if you get a cleaner version of the userdata partition.

I fully expect that if you were to do a chip-off forensic imaging process of a newer device, you would get the same garbled output as you would if you imaged /dev/block/mmcblk0.  So if you get newer devices, chip-off probably won't do you any good.  Can anyone out there confirm?  Once you've got the chip removed, it is difficult if not impossible to put it back in place.  Chip-off is a rather one-way method.

Note: I can do a screenshot demo of the above, or maybe even a video demo.   However, I currently do not have an Android 7.0 capable "hack-around" phone or tablet.  I had been using a Nexus 7 (2013) and a Nexus 5 as hack devices.  The Nexus 7 is no longer supported on new Android versions, and the Nexus 5 has ... seen better days.  Those were pretty cheaply manufactured phones and 3.5 years of daily use did little good.  So if you'd like to see some demos, consider clicking on the PayPal link on the right side and making a small donation to help offset the cost of a newer hack-around device.

See?  Not so long winded, huh?

  • Many newer devices likely include file-based encryption, resulting in garbled user data if you image the entire device
  • Use the mount command to find the right partition and you should be in good shape
  • Don't jump straight to chip-off.  You might end any real chance at imaging the userdata partition
Questions, comments, suggestions, or experiences?  Surprised at my brevity?  Leave a comment below, or send me an email.

Wednesday, March 22, 2017

Fun with Apktool

Or a potential headache

All blog posts to date
Opening night, my wife and I saw the movie "Logan" on the big screen.  I have to say, the movie was incredibly violent and it took a while for the shock to wear off.  But the shock has since worn off and I've had plenty of time to think about it, and I've come to a singular conclusion:  the film was outstanding.

The film focused on strong characters that I have grown to love.  Hugh Jackman first came on to the scene as Logan and Patrick Stewart first brought such elegance to the role of Charles Xavier nearly 20 years ago.  I have grown to love these characters, seeing all of the movies they are in, even that terrible embarrassment X-Men Origins: Wolverine.  "Logan" is amazingly emotional, dealing with the difficult topic of time; both Logan and Charles know the last tick of their clocks cannot be far away.  Charles, the man with the most powerful mind ever known, is losing his mind; Logan, with the unbeatable body, is losing his body.  They could simply cower away and live out the remainder of their lives in reclusion, but events happen which lead these two men to endure great sacrifice in order to help a girl they do not know in a desperate situation.

So you're probably wondering right now, why on earth am I talking about an awesome character-driven action film on a forensics blog?  Well, here goes.  In the film, Logan (spoiler alert) hacks a lot of things and people to pieces, and (spoiler alert) the X-Men franchise sometimes involves cloning.  In this post, we will be hacking around with apps and cloning apps.

OK, OK, OK, I'll admit, that's a pretty weak tie-in.  Truthfully, I just loved the film and wanted to talk about it.  So here goes.

Android apps are packaged as apk files.  These files are essentially zip files.  For a quick guide on Android app files, check out this previous post I wrote on reverse engineering apps.  

Apktool is a free, open source tool for decompiling and rebuilding apps.  Here's the main page.  The tool reverses the app's code to smali, it extracts embedded images and XML files, and it properly decodes the Android manifest.  It is an excellent tool for reverse engineering.

Now what is smali? Smali is reverse-engineered Android app code.  Android apps are written in Java.  The Java code is compiled into machine-readable code.  The guide I wrote on decompiling Android apps involves converting the app into a Java jar and then decompiling the jar.  This is a fine way to do it but is honestly not the most "accurate" way.  The most accurate way is to decompile the app code itself, and that app code is decompiled into smali, which is almost like assembly code.  Here is an excellent writeup on smali

Now understanding smali is a pain.  I'm not the best at it, which is why I decompile apps the way I do by converting the app to a Java jar and decompiling the jar.  If you want to learn some smali, here is a blog with some excellent posts that can serve as great starting point.

Apktool allows you do decompile an app for reverse engineering.  There also is now a tool which allows you to use the decompiled code for debugging an app.  The tool is called SmalIdea and it acts as a plugin for the Android Studio development environment.  I will not go into detail now on SmalIdea - that would be a detailed post in and of itself.

Apktool also allows you to rebuild an app from the decompiled output.  You can decompile the app, make some edits as you see fit, and repackage it.  Legal disclaimer:  you can reverse engineer an app for your own personal interests or understandings, but absolutely do not repackage an app and attempt to profit from it.  Do not distribute the repackaged app and absolutely do not sell it.  If you sell somebody else's intellectual property, that is intellectual theft.

So where does the topic of forensics come in play with Apktool?  Any tool that can be used for reverse engineering is useful for forensics.  So let's do a quick decompile.

In the film Logan, the main characters go on a road trip.  Anybody who has ever been on a long road trip knows highway rest areas can be a lifesaver.  So I downloaded a rest stop locater and reversed it.

I pulled the app off my Android device and renamed it on my local computer "restarea.apk".  Then I downloaded the newest version of apktool and renamed it "apktool.jar".  So here's the line to decompile:
java -jar apktool.jar d restarea.apk
Apktool is a jar so it must be run in Java.  The "d" means "decompile", and then you give it an app to decompile, or in this case, restarea.apk.  Once the tool runs, there is a directory called "restarea".

Within the restarea directory, there are three specific items of note:
  • AndroidManifest.xml: this is the Android manifest, describing the app, permissions, screens, and included files.  Here is the documentation for the manifest
  • res: this is a directory containing images and text files which are part of the app.  The app icon is in here, any image buttons are in here, and many hard-coded text values are in here.
  • smali: this is a directory containing all the decompiled smali code.
As an examiner, all of these can be useful.  Knowing the package name from AndroidManifest.xml will help you find data associated with the app.  Knowing text values will help you understand the behavior of the app.  And an understanding of the smali code will allow you to know the implementation of the app.
All useful.

Cloning an app
Apktool can allow you to edit and repackage an app.  Let's use that same rest area locater app.  First, let's change the package name around.

Here is the beginning of the AndroidManifest.xml file:

<?xml version="1.0" encoding="utf-8" standalone="no"?>
<manifest xmlns:android="" package="com.omecha.restarea">

The package name is com.omecha.restarea.  I edited that around to customize a rest stop finder for Logan.  Now the beginning of the manifest is as follows:

<?xml version="1.0" encoding="utf-8" standalone="no"?>
<manifest xmlns:android="" package="claws.omecha.restarea">

So now the app's package name is claws.omecha.restarea.  This will be notable later in the demonstration.

Next, I changed around the app's name as it appears in the loader.  In the file res/values/strings.xml in the decompiled directory, there is an entry app_name, which is as follows:

<string name="app_name">Rest Area Locator</string> 

I edited that line to the following:

<string name="app_name">Claws-Safe Rest Area Locator</string> 

After all, if a rest area is not safe for someone with claws, Logan should skip the rest area, right?

If I wanted to, I could have changed around the app's icon.  All the image files are in the res/drawable directories.  And if I really wanted to be adventurous, I could have gone into the smali directory and edited around the decompiled smali code to change functionality, but I'll admit I'm just not good enough at smali to do anything of significance.

Now, it's time to recompile the app.  Navigate back to the directory with apktool.jar and execute the following:
java -jar apktool.jar b restarea

The "b" stands for "build", and "restarea" is the decompiled and edited output.  Once the build is done, there is a directory called dist with a file restarea.apk.  That is the built apk.

It cannot be installed on an Android device just yet.  It needs a new app signature.  Just follow the instructions on this Stack Overflow post and the app has a new self-signed signature that allows you to install the app on your own device.

Then I installed the app and, well, check out these screenshots:


What we've got now is the original app and a cloned, or maybe I should say mutated, version of the app.

If you navigate to the device's /data/data/ directory, you see app data.  And in that directory, you see entires for both com.omecha.restarea and claws.omecha.restarea.  These directories store data associated with the apps.  More on the topic here.  Each directory has a databases directory with a database of user data, each directory has a shared_prefs directory with xml files, etc.  And if you create some user data in the com.omecha.restarea version, that data will not show up in the claws.omecha.restarea because these are two different apps.

And again I have to say, feel free to experiment around as I have shown here simply for personal study.  Absolutely do not steal somebody else's work and attempt to pass it off as your own.  That is dishonest and dishonorable.  And I should not have to say this but I will.  Do not make a modification like this and then attempt to make money off of it.  That is illegal.

What's the big deal?
So why does cloning an app matter as a forensic investigator?  That's the big question.  And here's the answer.

Let's say you are examining an Android device.  You run some automated tools at the device image and you find nothing of any real interest.  Those automated tools may look for data within specific apps.  For example, as I noted in my post on Facebook app forensics, the app has two different package names, com.facebook.katana and com.facebook.orca; the first is the main Facebook app, the second is the Messenger app.

Now let's say the user is an advanced user who has the knowledge to clone and mutate an app, or the user knows such an advanced user.  Let's say the Facebook app has been modified and cloned and is now renamed a different package name, like mutated.facebook.  That automated tool that is looking for Facebook data in com.facebook.katana or com.facebook.orca could go right past this mutated app and miss out on conversations.  Mutating an app is effectively a data hiding technique.

How do you find such data?  Just examine data in all third party apps.  Examine the databases and if you find something of investigative value, such as conversation messages or call logs, flag that app as interesting.  Examine the data closely.  You might have found an app you've never heard of, or you might have found a cloned version of a real app.

  • Apktool is an excellent tool for reverse engineering apps in order to understand functionality.  Learn some smali and there is no limit to your understanding of an app's mechanisms
  • You can use Apktool to mutate an app, changing package names, images, and even functionality
  • Mutating an app can be an effective data hiding technique.  Over-reliance on automated tools can lead to missing out on important data
Questions, comments, suggestions, or experiences?  Seen Logan?  Leave a comment below, or send me an email.

Saturday, February 25, 2017

Waze for Android forensics

Lots of Location Information

All blog posts to date
Many years ago, I spent an evening at my grandparents' house before taking off for a day-long there-and-back trip across state lines to my soon-to-be university.  The trip was just before I started grad school at the university and I was interviewing for a graduate assistantship, which I earned.  I had been to the campus a few times, but I can't say I was overly familiar with the turf and I had never driven there from my grandparents' house.  So my grandfather gave me an old US atlas of his.  An old Rand McNally US highway atlas that Wal-Mart published every year.  Remember those?  They would publish a new highway atlas each and every year in the off-chance that main street might get up and move between this year and next.

Anyways, I used that atlas to navigate.  I'm a natural with maps - I grew up backpacking and therefore relying upon trail maps - so I found my way there and back.  And of course, this was towards the beginning of the smartphone era, so I did not have a digital device for navigation.  Paper was just fine.

I kept that atlas around for other trips.  One year, a friend of mine and I drove from the Midwest down to Alabama and back for a football game.  It was an awesome trip, including a stop at the Louisville Slugger factory and museum and another at the Space and Rocket Center in Huntsville.  That friend of mine also is old-school like me, not needing an LCD screen to get from point A to point B.  I kept that same old atlas around for other road trips, for football games, skiing, and so forth.

Where is that old atlas now?  It's been cut up and turned into a Christmas present.  No, I'm not kidding.  My dad is a marathon runner, aiming to run a marathon in all 50 states, so my wife and I made a little scrapbook for him to document each run, photos and such, against a map of each state.  And why was I willing to sacrifice that atlas?  Because, you know, who uses an atlas anymore?

There are several maps and navigation apps out there for Android.  I find Waze to be such a novel app in that it is a combination of navigation and social networking.  Meaning app users report road incidents so other users can be aware of accidents, construction, roadkill, traffic jams, and other slowdowns.  Waze effectively crowdsources traffic information.

You can use Waze as a GPS navigation app, for communicating slowdowns, for sharing your location and trips with friends, and I've found it has an incredibly loyal following.  Waze-ers seem to never flip back to Google maps.  Point is, as a this app gives you, the forensic examiner, locations, times, and a social network.  That is gold for an investigation if the target uses Waze.

So I populated a phone with Waze, imaged it, and disected the data.  There's a lot of geo-location there, and it is quite easy to comprehend.  So ... here we go.

The package name for the app is com.waze.  So once you've got your image, check out the directory com.waze within the data app of the userdata partition.

The main file to check out is user.db - in that directory com.waze, not in any subdirectory.  The database has a bunch of tables.  I will highlight the ones of interest.  This is a SQLite database.  I did a post a while back on viewing SQLite databases.

First, the table PLACES.  This one stores places the user has searched for and selected as a navigation destination.  Here's the columns of interest:
  • name: Name of the destination, such as "Home", or "Safeway"
  • street:  Street address of the destination
  • city:  City of the destination
  • state:  State of the destination
  • country:  Country of the destination
  • house:  Apartment or other unit number
  • longitude:  Longitude, multiplied by 1,000,000.  Add a decimal accordingly
  • latitude:  Latitude, also multiplied by 1,000,000
  • created_time: Epoch time it was searched.
I was going to screencap the database, but it would not be worth much after I would black out all the personal sensitive data, which is all of it.  I'm not about to let the Internet know where I live, where I work, and when I go to where!

This is all plain text.  All you need to do is an epoch time conversion and you've got a listing of when each destination was searched for, exactly where on the planet it is, and the street address.  This table alone can be a goldmine for an examiner.

Next the table PEOPLE.  With Waze, you can connect people via Facebook, and then you can share your location and coordinate travel.  Here's the columns of interest:
  • waze_id: The Waze ID of the user in order to link to the right Waze user.  More on the device's Waze ID later.
  • facebook_id: Facebook's ID of the contact in order to link to the right person.
  • first_name: First name of the contact
  • last_name: Last name of the contact
  • create_time: Epoch time the contact was added
  • modified_time: Epoch time the contact was modified last
So far, Waze has provided your location search history and your contact history.

Next, the table SHARED_PLACES.  This table includes locations the user has shared, which may mean the location is of significance.  Really there are only a few columns of interest, so check out the created time, the place name, and the share time.  Pretty self explanatory.

There are some other interesting tables in the database.  Feel free to browse around and see if anything else is of interest.

XML Files 
Next up, check out the directory shared_prefs.  This includes some xml files.  I'll highlight two of interest.

First, the file com.waze.appuid.xml .  I previously mentioned the Waze ID.  Here it is.  Linking the Waze ID of one device in this XML file to another device in the user.db, table PEOPLE, indicates these two users know each other.

Second, com.waze.parked.xml.  Here's what mine looks like:

<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
    <string name="dest_lon">[REDACTED]</string>
    <string name="dest_name">
    <string name="dest_venueId">
    <string name="dest_lat">

When you finish a Waze trip, which should naturally end with parking the car, this file is created.  It stores where the car is and when the trip ended.  Nifty, huh?

Finally, go back up a directory and check out the file waze_log.txt.  This is a massive log file with some decent goodies.
There are geo-coordinates which represent different stops along the way on a trip.  I also found information about routes from point A to point B.  Of interest, there is a list of each route of highlights along the way - anything from airports to groceries to gas stations.  This may be of interest.  There are all kinds of businesses listed that are near the route - which may also be of interest.
This was a rather simple app study - and I did not go all too deep into the app data.  If you have a specific app you would like me to do a deep dive, let me know.  I may be up for it.  Additionally, the data in this post could be easily transformed into a simple forensic parser.  If you would like a simple Python script to parse all this data, let me know.  It shouldn't take me too long. 

Another blog
As an influential member of the mobile forensics community, I believe in promoting each other's work.  There is a blog from a few years ago that appears to still be valid today. Apps change and so sometimes findings for one version of an app are invalid when the app upgrades. These findings look good on current versions of the app. The blog was a capstone project for undergrad on this topic.  Check out this link for some excellent Waze work.

  • Waze stores a good amount of geo-history in easily accessible plain text, mostly in a single database
  • An XML file stores the last place and when the car parked at the end of a Waze trip
  • The waze_log.txt file has a lot of data and I've barely checked it out
Questions, comments, suggestions, or experiences?  Fun road trips?  Leave a comment below, or send me an email.

Thursday, December 8, 2016

Dirty Cow

A potential game changer in forensics

All blog posts to date

About two and a half years ago, I wrote a post on live imaging an Android device.  Based on the view stats I see on this blog, it seems like this post has a lot of popularity.

Fast forward some time and several operating system revisions, and the post is now quite obsolete.  The post relied upon TowelRoot, which is an exploit that has been patched for over two years.

Now what is not obsolete is the general method.  Data connection between the forensic computer and the device, exploit, imaging command.  These concepts are still the same.  The problem is, I don't have an exploit for new Android devices.

Dirty Cow
Enter cve-2016-5195, or "Dirty Cow".  The link in the previous sentence is to the official documented exploit.  In short, it is a Linux exploit that is also in the Android kernel.  It could potentially be used as a root vulnerability.

Several developers have released open source versions of Dirty Cow for Android, but all as proof-of-concepts.  To the best of my knowledge, nobody has released a version of Dirty Cow specifically for rooting devices.  I have tried several techniques on my personal phone and still no root shell.  The source is open in C and can be compiled using the Android NDK.  Now I personally have C experience, but the last time C was my primary language, the top selling phone worldwide was the Motorola Razr and Barack Obama was a little known senator from Illinois.  And the world still hated Tom Brady and the Patriots, so I guess some things never change.  Didn't I mention in my previous post that I'm officially a dinosaur?

If anybody gets Dirty Cow working on Android like Towel Root, meaning a do-all root program, then suddenly we as forensic examiners can use the live imaging guide to image any current Android device, or at least until Dirty Cow is patched and the patch is wide spread.

As I am using a Galaxy S6 running 6.0, I have not rooted my phone.  That may come as a shock to many.  The reason is I do not have a way to root my phone without tripping the Knox warranty bit.  I would like to keep that intact but still gain a root shell.

And of course, where there are forensic implications, there are also security implications.  If I can image anyone's phone, then so can anyone else.  And anyone can access and take privileged information from a device.

Community Work
So here comes the point of this post.  Who out there is working on Dirty Cow or other new exploits?  If anyone reading this is interested in Android forensics and is working on gaining root shells, I'd sure like to hear about it, whether you are using Dirty Cow or something else entirely.  If you are able to, please share.  I am happy to collaborate or point you in the direction of someone who can collaborate as well.

And, have you had any success with Dirty Cow or any other current exploit?  If so, how did you do it, what device, any troubles, etc?

I personally would be very interested in getting an exploit for newer Android phones up and running..  The purpose here is for forensic research so I can share with the digital forensics community community continuing results.


  • TowelRoot is obsolete.
  • Dirty Cow is a a possible way to gain an equivalent root shell on newer devices.
  • Collaboration?
Questions, comments?  Any research you wish to share?  Leave a comment below, or send me an email.

Saturday, June 25, 2016

Interpreting data from apps

Lots of apps means lots of data

All blog posts to date
In Android, just about all the data you care about will be app data.  Text messages?  App data from the SMS app.  Phone logs?  App data from the phone app.  Facebook chats?  App data from the Facebook app.

Now I think of myself as being a young guy.  At least, I'm not old, or too old.  I am a millennial.  So you could imagine the surprise when I heard the following comment from a college-aged family member:
Mark, you still text?  You are such a dinosaur.

Yes, I am a dinosaur for still texting.  Now I can remember literally laughing out loud when I first received a text from my father.  He had finally graduated from placing a phone call for even the simplest of messages to convey to sending a short message over SMS.   Fast forward a few years and I am a dinosaur for not graduating beyond texting.  But graduating to what?

Well, that college-aged family member said he and all his peers use apps to message each other.  Facebook, WhatsApp (which is now owned by Facebook), and I'm sure others.  C'mon, I asked him, what's wrong with texting?

And why use Facebook instead of texting?

Can you send photos over these apps the way I do with MMS?

And what exactly is WhatsApp?

Now beyond the young kids who dance on their elders' fossils for using a technology which my elders only recently began using (poorly), there are others who, for better or worse, use apps for communicating.  For a dramatic example, the terrorist organization ISIS famously uses the app Telegram for spreading propoganda.  Now you may think that the owners of Telegram may consider it a responsibility to shut down ISIS usage.  And Telegram has in fact made an effort to ban terrorist usage of their app, but that was only after laughing off the notion with the following line: I propose banning words. There’s evidence that they’re being used by terrorists to communicate.

Also sadly in the news was the app Kik  In the recent tragic murder and kidnapping at Virginia Tech, the perpetrators used the app Kik to lure in their prey.  The story is horrifying on so many levels.

Now what about other apps?  Games, sharing apps, social media, etc.  Might those apps store some significant data too?  This post is all about how to parse and interpret data from third-party apps.  Why is this significant?  If you get all the SMS and call logs and other traditional evidence, you may have missed the device owner's primary method of communication.

Now this post will not be a specific how-to-parse-this-app post.  Instead, this is a generic guide for parsing that may be of help.  I hope to convey methods apps store data and how to access and read this data.  I believe the challenge of parsing and interpreting app data is or will soon become more tedious than imaging devices.

Where do apps store data?
First, Android security prohibits users from accessing the userdata partition, which is where all apps store their data.  (Some apps may also store some data on the SD card, but this is "unprotected" data.  Not the "good stuff.")  You either need an image of the device (and you can create an image using my post on live imaging an Android device) or you need root access.  In this post, I am working from an image of a device.

Android by default stores user data in the /userdata partition in the directory /data.  The below screenshot is from a screenshot of FTK Imager looking at the data directory.

You'll see that within the data directory are directories containing package names.  The directory air.WatchESPN stores user data associated with the WatchESPN app.  The directory stores data associated with the YouTube app.  The directory will story web history and other data associated with the Chrome browser.

What kinds of data?
By default, most user data is stored in SQLite databases.  For a writeup on viewing SQLite data, check out my previous post on SQLite databases.

Most apps use SQLite in some fashion or another.  And if the app you are trying to parse stores all its data in SQLite with no encryption of any kind, you are good to go.

Other types of data can also be found.  The following is a screenshow of the YouTube app's storage:  

All of the directories above store data associated with the YouTube app.  The databases directory will contain SQLite databases.  The directory shared_prefs stores XML files which may be interesting or not.  Depending upon the app, the XML files may store data about the user, such as usernames or maybe even passwords if the developer has a poor grasp on security.  The XMLs can also be pretty much useless.  XMLs can be opened in any text editor of choice.

There also is a "files" directory.  This directory can store anything.  A developer can store images, videos, text, or even more databases in the files directory.

So if your app stores only databases in a nice SQLite format, some user information in XMLs in shared_prefs, and some images or other interesting files in the files directory, you can easily interpret all of this data.  But what about challenging data?  I will highlight two challenges to consider with app parsing.

Many apps use encryption to store sensitive data.  In some cases, you open a SQLite database and are parsing through and you get to a table with nothing but random-looking junk stored in rows.  If you come across such a finding, you may have found some encryption.  This is an example of encrypted content within a database.  And if you find such content, you will need to find a way to decrypt the data to make any sense of it.  I am no crypto expert, so I would consult one if I came across such a finding.

And a side note.  If you come across a string that looks like the following:

you have come across Base64 encoded text.  This is not encryption and can easily be decoded.  Base64 encodes anything into a random set of uppercase, lowercase, numbers, and some symbols, and if padding is required, the string ends with equals signs.  Here is a Wikipedia page on Base64.  You can come across Base64 in databases, XML files, URLs, or just about anyplace.

Some apps also encrypt their databases entirely.  WhatsApp, for example, encrypts their entire database.  If you come across an app that you know is storing a good amount of data on the device and yet you cannot find a database but you find entire files of random-looking data, you may have found encrypted databases.  Again, consult a crypto expert.

Non-standard data storage
App developers are developers, and developers like to develop things.  What do I mean?  I mean that developers often get tired of using built-in functions, like SQLite, and so they choose to implement a different database format, or they make their own.  I have plenty of jokes to make at my engineer friends' expenses about over-engineering everything.

So while SQLite is a nice and easy format to parse, there are other database formats out there for Android.  Here is a list of ten known and available non-SQLite database formats.  If you come across an app with a non-SQLite database, you will need to find a way to interpret all the data.  If there is not a parser available, you can use a hex editor to simply view the database and make sense of it, or you can (or have a developer) write a parser so that next time you come across such a database, you will be ready.

So big picture.  What do you do when you find an app on a device and you are unfamiliar with it?  Here is a list of steps you can do to make sure you get all the possible data.
  • Examine all the data files.  View SQLite databases.  Open XMLs in a text editor.  Open unknown files in hex editors.  View any media.  Make sure you view everything.
  • Check out the file /data/system/packages.xml.  This is a file which stores information about all apps installed on the device, including device permissions.  See any permissions that stand out?  If you see camera permissions, be on the lookout for photos associated with the app.  In a previous post, I detailed the file.
  • Reverse engineer the app.  Look at the source.  It may help you understand what the app does and how it stores data.  Here is a previous post on reverse engineering.
  • Once you have made sense of the data, report it in a standardized and readable format.
  • If you think you may come across this app again, consider writing a program to parse through data based on your findings so you can do this process automatically.
Resources available
There are plenty of resources available for interpreting data from the diverse apps out there.  I will list out a few.
  • Mobile forensic tool vendors.  I was recently at Mobile Forensics World in South Carolina.  There were many vendors presenting similar information to this post.  Everybody in the mobile forensics community is dealing with all these third party apps.  For example, the company Magnet Forensics sometimes releases findings of different third party apps.  Here is an excellent writeup on data within the app Skype.
  • dex2jar.  In my post on reverse engineering, I show how to use this program to reverse an app to a Java jar.
  • Java Jar decompilers.  With the Java jar from dex2jar, a decompiler can interpret the jar as Java source for reverse engineering.
  • Wireshark.  Especially with chatting or social apps, you may need to understand data coming over the air to the device.  Wireshark can help you capture data in transit.
  • JEB.  An app decompiler.  This is most definitely not free software.  If you need to decompile and debug an Android app to see from the "app's perspective" how data on the device is created, JEB can do the trick.


  • There are so many apps out there which store important data on the device.  If you only look at SMS and call logs, you may miss the most important conversations on the device.
  • Apps store data in the userdata partition.  You need either an image of the device or root access to get at it.
  • Data can be stored in challenging format.  If you come across encryption and you are not a crypto expert, you may need to call one in.
  • There are resources out there.  Everybody in the community is dealing with this challenge.
Questions, comments?  Any other dinosaurs out there?  Leave a comment below, or send me an email.

Saturday, February 27, 2016

Some non-root methods to learn about a device

Sometimes you cannot image or root a device

All blog posts to date
This post will be a bit of a change of gears.  Most of the rest of my posts involve imaging the device in question and examining the image.  And of course, imaging requires three things:  
  • Data connection between the device and the computer
  • Exploit
  • Imaging command
Now what if you cannot or choose not to exploit (root) your device?  This is a real situation.  So long as you have the ability to adb shell into the device, there still are some ways to get useful data from the device.  This post is all about some simple ways to gain some insight into the device.

Determine if the device has been compromised
A common fear with Android devices is that the device can be rooted without the user knowing.  This is a legitimate concern, and it can be addressed with an adb shell to the device.  Also this takes a couple little pieces of knowledge:
  • The system partition, mounted at /system, is mounted read only
    • Normal users cannot modify anything in this partition without a root exploit
  • The root binary, su, is typically found in the system partition
  • Which all means that if the device is compromised, there is most likely evidence of it found in /system

Shell into the device and cd to /system.  Then do an ls -al.  The -l part of ls will get a listing of all the files and directories at the /system directory including last modified times, and the -a will list any hidden files so you do not miss anything.  Here are the results from my Nexus 5:

shell@hammerhead:/ $ cd /system
shell@hammerhead:/system $ ls -al
drwxr-xr-x root     root              1970-11-15 06:44 app
drwxr-xr-x root     shell             1970-11-15 06:44 bin
-rw-r--r-- root     root         4065 2015-04-17 13:34 build.prop
drwxr-xr-x root     root              2015-06-25 22:10 etc
drwxr-xr-x root     root              2015-04-17 13:34 fonts
drwxr-xr-x root     root              2015-04-17 13:34 framework
drwxr-xr-x root     root              1970-11-15 06:44 lib
drwx------ root     root              1969-12-31 19:00 lost+found
drwxr-xr-x root     root              2015-04-17 13:34 media
drwxr-xr-x root     root              2015-04-17 13:34 priv-app
-rw-r--r-- root     root        89346 2015-04-17 13:34 recovery-from-boot.p
drwxr-xr-x root     root              2015-04-17 13:34 usr
drwxr-xr-x root     shell             2015-04-17 13:34 vendor
drwxr-xr-x root     shell             2015-06-25 22:10 xbin

As the timestamps show, most directories were last altered 2015-04-17, which is when I last flashed a version of Android onto my device.  Yes, I am behind the times.  Intentionally.  I like Android 5.1.1 and I'm sticking with it for now.

But you will also see some oddities.  Namely two directories that were updated two months later on 2015-06-25.  What is going on with etc and xbin?  Let's find out.

So I cd to etc and do another ls -al.

shell@hammerhead:/system $ cd etc
shell@hammerhead:/system/etc $ ls -al
-rw-r--r-- root     root            2 1970-11-15 06:44 .installed_su_daemon

-rw-r--r-- root     root         1472 2015-04-17 13:34 DxHDCP.cfg
-rw-r--r-- root     root       277097 2015-04-17 13:34 NOTICE.html.gz

(A bunch more files ....)
-rw-r--r-- root     root           38 2015-06-25 22:10 resolv.conf
(A bunch more files ....)

drwxr-xr-x root     root              2015-04-17 13:34 updatecmds
drwxr-xr-x root     root              2015-04-17 13:34 wifi
shell@hammerhead:/system/etc $

You will see that the file resolv.conf, which can only be read and written but not executed (explanation found here), was last modified 2015-06-25. So what is in that file and should I care?

I also can see that the file is only 38 bytes.  Small file.  So I just cat it.

shell@hammerhead:/system/etc $ cat resolv.conf                                
shell@hammerhead:/system/etc $

The contents of the file are pretty simple. For an explanation of what "nameserver" means, quite literally consult Google.

So on the one hand, you have a file which basically just says use Google for DNS, so nothing of real significance.  On the other hand, why was that file modified when it was?  If you remember, at the same date and time that the resolv.conf file was modified, the /system/xbin partition was modified.  Let's cd to that and get an ls -al.

shell@hammerhead:/system/etc $ cd ../xbin
shell@hammerhead:/system/xbin $ ls -al
lrwxrwxrwx root     root              2015-06-25 22:10 [ -> /system/xbin/busybox
lrwxrwxrwx root     root              2015-06-25 22:10 [[ -> /system/xbin/busybox
lrwxrwxrwx root     root              2015-06-25 22:10 acpid -> /system/xbin/busybox

-rwxr-xr-x root     root      1095836 2015-06-25 22:10 busybox

-rwxr-xr-x root     root        75364 1970-11-15 06:44 su-rwxr-xr-x root     root        75364 1970-11-15 06:44 sugote
-rwxr-xr-x root     root       157412 1970-11-15 06:44 sugote-mksh

lrwxrwxrwx root     root              2015-06-25 22:10 zcat -> /system/xbin/busybox
lrwxrwxrwx root     root              2015-06-25 22:10 zcip -> /system/xbin/busybox
shell@hammerhead:/system/xbin $

So what does this all mean?  It means that on 2015-06-25 at 22:10 EST, I installed busybox on my device.  Busybox installed all these logical links at /system/xbin which point to busybox, and busybox additionally changed the /system/etc/resolv.conf to point to Google.

Additionally, you can see that on November 15, 1970, I rooted my device as you can see the su binary.

Wait, what?  In 1970, neither the device nor me existed.  Google didn't exist, cellular connectivity didn't exist, computers used punch cards as input, and bell bottom jeans were hot fashion.

No, most likely, I rooted my phone in June 2015.  However, I rooted my phone via recovery mode, which does not use the Android clock, so all activity appears to take place in 1970.  I hate to disappoint you all, but I am in fact not a time traveler.  Additionally, this explains the file /system/etc/.installed_su_daemon as seen above, also modified in 1970.

So quick takeaways.  If you are worried that your phone might be compromised, do an ls -l in /system and see if anything looks like it was modified at strange times.  Additionally, see if /system/xbin/su exists.

Now some caveats:
  • The system clock can be changed.  Easily.
    • A smart attacker can change the clock around and do changes so that a simple analysis as seen above will not uncover changes made.
  • The su binary need not be called su and placed in /system/xbin
    • It could be elsewhere and called something else.  As long as it has the right permissions, the file can be placed anywhere on the device with execute privileges.
As always, if you are concerned there is some advanced attacking going on against your device which may employ some clock modification or a hidden root, you can contact me for further assistance with your device.

See what apps have been installed and if there are any you do not recognize
You install apps on your phone and you know what you've installed.  Or you install so many apps that you forgot what you've installed but the fact is you did personally install the app and you can account for it.

But what if somebody else gets their paws on your phone and installs something? We've all read articles about scares over spyware on phones, and we've all seen those movies where the bad guy gets the good guy's phone and installs something nasty which lets the bad guy listen on and everything the good guy is doing.  The truth is the Android development environment can be exploited to allow such apps to run.

So what if you think somebody has "tapped" your phone?  There is a good chance that if your phone has not been rooted and yet somebody put something on your phone, the little gift they left for you is an app.  It may be hidden, but it can be found with some simple command lines.

I'll give a quick and simple method which gets a list of all user-installed apps and then another method which tells details of all apps on the device.

So the quick way first.  Adb shell into your phone and type the following command:

pm list packages -3

pm is a command which stands for "package manager".  It does just as you may think - it manages packages, or apps, on the device.  The command above lists all third party apps, or user-installed apps.  The output from my phone is as follows:

shell@hammerhead:/ $ pm list packages -3
... a bunch more third party apps ...
shell@hammerhead:/ $

If any of the apps listed look like something you do not recognize, perhaps you should look into it.  I personally recognize all the apps installed.  stericson.busybox is a shortcut way to install busybox.  I care about the weather, hence a weather app.  My wife and I went to go see Deadpool last week, hence the Fandango app.
Now let's say you want to look into a specific app.  You can use the dumpsys command, which accesses all types of system logs.  For example, let's do a look into the WatchESPN app.  I may be a massive nerd, but I love my sports.
I enter the following line ...

dumpsys package air.WatchESPN

And I get the following info about the WatchESPN app
shell@hammerhead:/ $ dumpsys package air.WatchESPN                            
... a bunch of developer related activity at the top ...
  Package [air.WatchESPN] (2f619c56):
    userId=10111 gids=[3003, 1028, 1015]
    pkg=Package{2ab5f4d7 air.WatchESPN}
    versionCode=2100103 targetSdk=22
    applicationInfo=ApplicationInfo{36a34671 air.WatchESPN}
    supportsScreens=[small, medium, large, xlarge, resizeable, anyDensity]
    timeStamp=2015-10-31 08:03:15
    firstInstallTime=2015-10-10 15:09:36
    lastUpdateTime=2015-10-31 08:03:16
    signatures=PackageSignatures{155cf1c4 [2bd4d8ad]}
    permissionsFixed=true haveGids=true installStatus=1
    User 0:  installed=true hidden=false stopped=false notLaunched=false enabled=0
shell@hammerhead:/ $

So what does all this mean?  Let's break down some important highlights.
  • codePath=/data/app/air.WatchESPN-2
    • The apk is in the /data/app/air.WatchESPN-2/ directory
  • legacyNativeLibraryDir=/data/app/air.WatchESPN-2/lib
    • Any native libraries are in /data/app/air.WatchESPN-2/lib/
  • dataDir=/data/data/air.WatchESPN 
    • Data associated with the app is stored in /data/data/air.WatchESPN - though you need to be root to access that data
  • firstInstallTime=2015-10-10 15:09:36
    • I installed the app in October 2015.  I'm guessing it was to watch this game.  My Bowling Green Falcons put on a show on offense.
  • lastUpdateTime=2015-10-31 08:03:16
    • The app was last updated on Halloween 2015.  The update must have been scary.
    • The app was installed by the standard method, not by ADB or other manual methods
  • grantedPermissions
    • The app has the following permissions:
      •       android.permission.INTERNET
      •       android.permission.READ_EXTERNAL_STORAGE
      •       android.permission.READ_PHONE_STATE
      •       android.permission.ACCESS_NETWORK_STATE
      •       android.permission.WRITE_EXTERNAL_STORAGE
      •       android.permission.ACCESS_WIFI_STATE
So this app is obviously legit.  And if this information looks awfully familiar, it is because the same information shows up in the packages.xml file.
You can also type the following command:
dumpsys package

and you will get a similar listing for every app installed on the device.  Now I would recommend catting this all out to a file.  The way to do that is get out of your adb shell and cat the output to a file on your computer.  It would look like the following, and note that I am shelled into my computer, not my phone:
adb shell dumpsys package > dumpsyspackage.txt

You will have a file on your computer called dumpsyspackage.txt with a full listing of every app on your device.

Now let's say I found something in the listing for the WatchESPN app that bothered me. I may want to reverse engineer the app.  I can pull the app without root access.  Here would be the line to pull the app onto the local computer:
adb pull /data/app/air.WatchESPN-2/base.apk

Based on the output above, the app is in the directory /data/app/air.WatchESPN-2/, and I happen to know the apks for third party apps in Android 5 get installed in that directory as the file base.apk.  The above command pulls that file to your local computer.
Continuing with that nifty dumpsys command ... 

Fun with dumpsys
As I said before, dumpsys is a command which accesses system logs.  It can tell you a whole lot more than just what apps are installed.
Now if all you want to do is obtain an entire system log, do a similar command as the previous command to dump all package logs to a single file.  Type the following command ...
adb shell dumpsys > dumpsys.txt

... and go grab a drink because it will run for a while.

The resulting file dumpsys.txt will be quite large, unorganized, and difficult to prod through.  So let's get a finer view of it.  ADB shell into the device and type the following:
dumpsys -l

I get the following output:

shell@hammerhead:/ $ dumpsys -l
Currently running services:
  ... a bunch more services ...
shell@hammerhead:/ $   

It is a list of all running services that can be dumped.  So let's say I want to find out about my wifi.  I can enter the following command from the computer, similar to previously dumping all app info.
adb shell dumpsys wifi > dumpsyswifi.txt

And I will get an output about what WiFi I am connected to, packet usage, etc.  You can do the same for any of these running services.  Want to know what your bluetooth is up to?  You know what to do now.

dumpsys is a good tool to play around with and it works on live running devices.  This is not something you can do to an image.  An image is just a file, a dead, flat file.  A device is a live, running computer that you can interact with, or in the above cases, interrogate.


  • There are some ways to get useful information from a live device, even without rooting or imaging.
  • Looking at files in your system partition by timestamp can be an indicator if your device has been compromised, though remember that timestamps can be faked with relative ease.
  • Dumpsys is a useful tool.
  • You can use dumpsys to learn about what apps are installed on a device, and you can use an adb pull to remove the app and examine if you wish.
Questions, comments?  Sports memories?  Leave a comment below, or send me an email.