Categories
Technology

Google Glass, Eye Love You

Having done a bit of research on what apps people are starting to develop for Google Glass, I stumbled on word of a recent Glass Hackathon where, among other things, an app was built to measure attraction:

It uses an eye tracker, mounted on the Glass frame, to measure the pupil dilation of the wearer. The idea is to measure someone’s arousal, giving an instant measure of how attracted they are to the person they are looking at.

Initially I misread this and thought it was built to track pupil dilation in someone else’s eyes, not the wearer’s. The thought of the proximity you’d have to have between your face and that of the person you’re analysing amused me greatly.

It did remind me of the sort of advice you see bandied around on the internet – “Dude if her pupils are dilating, she’s totally into you”. Dude, if you’re staring into each other’s eyes close enough to observe the dilation of her pupils, she’s probably into you.

Attraction

 

Lastly, on a more serious note, tracking the wearer’s pupil dilation has already been patented by Google, with some people suggest it could be used to gauge your emotional response to advertising.

Categories
Technology

How to remove security from a PDF

Source: https://www.flickr.com/photos/julajp/5638996854/
Photo by JuliaJP CC BY

I recently sat through an eLearning course which involved reading through a 1300-page PDF file. Instead of deforesting the Glasgow area to provide the requisite amount of paper for printing, I thought I’d read and annotate it on-screen (something I wouldn’t recommend unless it’s to save the environment)

Annoyingly, annotation is prevented on PDFs with security applied, so I had to find a way around it.

Firstly note that if you Google it, there are several tools and methods for removing security from a PDF, but they are a slow, frustrating hassle. I wasted a lot of time on crazy methods involving re-printing the whole thing to virtual PDF printers, etc – trust me the method below is the only sane way to go about it.

So… how do I remove security from a PDF?

  1. Open your PDF in Google Chrome
  2. Save the PDF from Google Chrome

Tadaa. Unbelievably simple, but very useful.

Categories
Programming SAS Technology

SAS Dashboard – Fixing the “too many dials” issue

There’s an interesting feature of SAS BI Dashboard that caught me out when trying to put together some KPI gauges.

We wanted a set of dials that would show us the counts for several types of public security offences. Naturally we configured indicators using count as the measure, but found that the dashboard was showing an indicator per record in our system:

kpi1

 

With a bit of faffing around, I eventually found that this was due to the columns I had selected from my data. Specifically, the “uniqueID” column. This led to my big discovery:

The number of dials = The number of unique combinations of column values

In other words, because I had selected a column where the value was always unique, I got a dial for every row in my table. If I selected just the “category” column, I got the aggregated view I expected:

kpi2
Perhaps this just shows my SAS Dashboard naivety, but I thought I’d document it anyway.

 

Categories
Programming SAS Technology

SAS – How to Export/Import packages

It’s also possible to import/export metadata with the Wizard equivalent

My team and I have been developing a solution which involves a degree of SAS reports and related metadata. I set up a scheduled, automated backup of our information maps, reports, etc for posterity, and out of general paranoia. For this I used SAS’s command line export and import capabiltiies, which probably weren’t designed to be used that way, but which turned out to be really useful.

It took a wee bit of trial and error, so I thought I’d document it here.

(Handy reference link)

Note: If you’re puttying into your SAS server, make sure that your putty session has “Enable X11” ticked.

 Export

(Ignore any new lines in the text below – I’ve added those for readability)

/usr/local/SASHome/SASPlatformObjectFramework/9.3/ExportPackage -host “mysasmachine” -port 8561 -user myuser@saspw -password mypassword -package “myPackage.spk” -objects “/Shared Data/mySourceFolder(Folder)” -includeDep -subprop

  •  includeDep means that all objects that the export depends on are also exported
  • You can also specify “-types” with the types of files you wish to export
  • Without specifying “(Folder)” on mySourceFolder, all files will be exported “flat” i.e. without their folder hierarchy

 Import

 /usr/local/SASHome/SASPlatformObjectFramework/9.3/ImportPackage -host “mysasmachine” -port 8561 -user myuser@saspw -password mypassword -package “myPackage.spk” -target “/Shared Data/myTargetFolder(Folder)”  -subprop myPackage.subprop

  •  Without specifying “(Folder)” on myTargetFolder it would create a new folder with the name of the old parent folder in the new parent folder (e.g. /myTargetFolder/mySourceFolder)
Categories
Programming SAS Technology

Converting custom date formats in SAS Information Map Studio

Background

Let’s assume you have some dates in a custom format:

  date20131007 month102013

SAS Reports need to be able to a) present dates in a human readable format and b) understand dates to allow filtering and other funky stuff.

For that reason we need a way of translating these custom dates into SAS dates.

Step 1 – Get an Information Map with a date field

Use an existing one, or see http://support.sas.com/kb/35/471.html for more information on creating an Information Map.

Step 2 – Edit the Expression of your date field

  1. Open the Properties for your date field.
  2. Then on the Definition tab, click “Edit” in the “Expression Settings” section.

Step 3 – Magic

  1. Change the Type to Date so that SAS can treat it as a date field from now on.
  2. Then change the Expression Text to something like this:
input(substr(<<mytable.mydatefield>>,5,8), yymmdd8.)

wat

What’s happening here is that we’re translating the custom date string into a SAS date using what’s known as an INFORMAT.

SUBSTR (string, 5, 8) – Takes a substring from the given string, starting at character 5, with a length of 8 characters. In other words extracting the string date20131007 month102013

INPUT (string, yymmdd8.) – Takes a string and interprets it using the informat “yymmdd8.”. That informat is provided by default with SAS. What we’re doing here is saying to SAS “Here’s a string, but I want you to start treating it as a date. So that you know which part is the year, and which part is the month etc, use this informat as a guide”. Then SAS can know that dd = 07, mm = 10, and yy = 2013.

We’ve effectively translated a string in custom format into a SAS date.

My mind is blown. Now what?

Interpreting the dates as dates means we can now make it human-friendly in our reports, and also allows us to do some excellent SAS-native date filtering.

Formatting

What we specified earlier was an INFORMAT – in other words an interpretation format. What we can do, now that the date is stored as a SAS date, is specify an OUTPUT format, so that we can represent the date in a variety of ways in our reports.

  1. Go back to the Properties dialog for your date field
  2. On the Classifications tab is a “Formats” section. Change the “Format Type” to “Date/Time” and look at the available formats.

Selecting a format will take your date and represent it as a different string depending on the format you choose. Some examples:

Format Output
DATE 07OCT13
DAY 7
WEEKDAY 1
MONNAME October
DDMMYY 07/10/13
DOWNAME Monday

Filtering

We can also now use SAS to filter dates in a very cool way. For example I can now filter all records which were created in 2013, or all records created after a certain date, or on a certain date, etc.

  1. Create a new Filter and choose your date field as the Data Item.
  2. Then set your Condition to “Year to date” – this will filter all your results to only show ones where the date falls between 1 Jan 2013 and today.
  3. Click OK

A note on filtering

It’s always preferable to apply a filter at the Information Map level, rather than typing in a manual filter when you’re creating your Web Report. A filter on the Information Map will mean the data is filtered at the source, rather than decoding a bunch of information and only filtering once we get to the report level.*

* My understanding is that this is only with certain databases. Some allow optimisation by passing through filtering into the queries they make against the source databases. Still a good practice if you can do it.

Categories
Programming SAS Technology

How to restart SAS server

Photo by nimCC BY
Photo by nim CC BY

More detailed instructions can be found here, but below are the steps I use.

Note – All instructions assume the SAS is installed on a linux box, in /usr/local/, since that’s where it was installed on my machine.

Stop all JBoss server instances

/usr/local/jboss-5.1.0.GA/bin/SASServer1.sh stop

Stop SAS processes

/usr/local/config/Lev1/sas.servers stop

Start SAS processes

/usr/local/config/Lev1/sas.servers start

(wait a couple of minutes)

Start JBoss server instances

/usr/local/jboss-5.1.0.GA/bin/SASServer1.sh start

(wait about 5, 10 minutes)

Categories
Programming SAS Technology

Creating SAS Web Reports from an Oracle data source

SAS Visual Analytics
Misleading screenshot of SAS Visual Analytics – but it looks much sexier than Web Report Studio, doesn’t it? 🙂

Here’s a sparse set of instructions for the steps you need to take to configure SAS to access data from an Oracle data store and surface it through SAS Web Report Studio (as an example).

Note that once it’s in a SAS format this can be used in any SAS product, and therefore means you can do all sorts of analytics on it. Hence the VA screenshot.

Caveats – Please note the purpose of this is not to break down every area step by step, but more to highlight the various areas that you’ll need to configure to get it all working together. See this as an orchestrator’s guide – I’ll try to document the areas that are necessary for a full orchestra, but I won’t tell you how to play each instrument! As such, these steps assume a working Oracle database setup, as well as the necessary SAS servers ready and waiting to be used.

I might go into more detail for each particular step in later posts.

Management Console

  1. Open Management Console

SAS User

First off choose a SAS user who will access the Oracle data (SAS Demo User).

  1. Go to User Manager
  2. Right click a SAS user
  3. Go to Properties > Accounts
  4. Click “New”
  5. Create a new Authentication Domain called “MyAuthDomain” and enter your Oracle RDBMS main user’s login details (e.g. myUser)
  6. You’ve now tied a SAS user to an Oracle user for a specific domain

Server

Next we create a server to represent where all the data is coming from (i.e. your MIE machine)

  1. Right click Server Manager
  2. Select “New Server”
  3. Select “Oracle Server”
  4. Enter a name (myserver)
  5. Set the “Associated Machine” to your SAS machine
  6. Set “Path” to the string you would use to connect to it if you were using mysql
  7. In other words, <ORACLE MACHINE>:<PORT>/<SERVICE NAME>
  8. e.g. ‘myoraclemachine.domain:1521/myservicename’
  9. Important – use single quotes around the whole thing
  10. Change Authentication Domain to “MyAuthDomain”

New Library

Now create a Library to represent a certain subset of your data – it’s like a logical grouping, and the library will be a lot more publicly accessible so the name must be good.

  1. Expand “Data Library Manager”
  2. Right Click “Libraries”
  3. Select “New Library”
  4. Select “Oracle Library”
  5. Choose a name (mytbls)
  6. Set the location to /Shared Data/<new folder of your choice> (create a new folder for your tables)
  7. Choose “SASApp”
  8. Set “Libref” to what your publically accessible name should be – it can only be 8 characters!
  9. Set “Database Server” to your server name (myserver)
  10. Set “Connection” to myserver (should be there already)

Register Library Tables

All you’ve done is create an empty library, pointing at your server. You need to actually choose the tables you want to use in this library.

  1. Click “Libraries”
  2. Right click your new Library (mytbls)
  3. Select “Register Tables”
  4. A dialog will appear telling you to log into your SAS machine… enter your SAS user details, NOT your oracle user details (always keep a close eye on which machine it’s asking you to log in to – other times it will ask for oracle credentials)
  5. Choose whatever tables you want to register in here

SAS Information Map Studio

Next we need to set up an Information Map to expose the view of the table(s) we want to use in reports.

  1. Open Information Map Studio
  2. Expand SASApp in the left hand side column to see your server
  3. Expand your server
  4. Choose any table or tables you want to use in your Information Map
  5. Choose any fields from those tables
  6. IMPORTANT – SAS will by default set the width of those fields to 32kb characters. Give these a more sensible limit otherwise this will break Web Report Studio (and potentially other things)
  7. Add any filters you want on those fields
  8. Make any Joins between tables you need
  9. etc
  10. Save your Information Map to Shared Data

SAS Web Report Studio

Now we have our data view exposed, we can build reports off of it.

  1. Open Web Report Studio – http://mysasmachine.domain:8080/SASWebReportStudio/
  2. Open the Report Wizard
  3. Select the Information Map you created as your data source
  4. etc etc
  5. Save the report
Categories
Programming Technology

Recursively list all files in Windows

 

You’re probably familiar with dir, the windows command which lists all files in a certain directory.

If you ever find the need (as I have) to recursively list every file in a directory, as well as all its sub-directories, you can do this by setting a couple of parameters on the dir command.

dir /s /b

Warning: This will output a lot of stuff, so I’d recommend piping the output to a file.

dir /s /b > output.txt
Categories
Programming Technology

Showing command history in UNIX with timestamps

A handy little command for checking the history of all commands executed through the UNIX command prompt for that user.

history

This will return you a numbered list of commands, oldest first.

Note that by default, “history” will NOT show you timestamps, which can be very handy if investigating activity on a certain machine.

To show history with timestamps, you’ll need to set up an evironment variable to store the format that history should be output in. Simply execute the following command.

export HISTTIMEFORMAT='%F %T '

Any subsequent call to history will show you the date and time.

Categories
Programming Technology

How to fix Oracle TNS Listener Issues

 A few times over the course of working with Oracle, I’ve come across environments which had issues making connections to databases set up across a network. These were failing with the error “ORA-12514: No TNS Listener”.

I eventually found that to fix this, you need to follow the steps below:

  1. Install the Oracle Data Access Components (ODAC) on the machine attempting to make the connection
  2. Navigate to C:\app\<username>\product\<oracle_version>\client_1\Network\Admin\Sample (note that this is a Windows 7 path – it will differ for other OS’s)
  3. Copy tsnames.ora into the directory one above it
  4. Open it up and replace all the values in angle brackets with the correct values for your set up:
<data source alias> =
 (DESCRIPTION =
 (ADDRESS = (PROTOCOL = TCP)(HOST = <hostname or IP>)(PORT = <port>))
 (CONNECT_DATA =
 (SERVER = DEDICATED)
 (SERVICE_NAME = <database service name>)
 )
 )

Best of luck!