Google Sky Map Review

I finally found the ultimate app out there for Android - no surprise its from Google!
You just point the phone at the sky and it shows you the names of the heavenly objects on screen!!! Stars, planets, constellations, galaxies even! Also, in case you cant find it, you can type the name, eg: moon, and voila, it guides you to which part of the sky that object currently is.

And there is a new feature called time travel, whereby you can view the sky forward or backward in time - so u can see historic eclipses or plan your future ones.

Another nice thing is that, though it uses GPS to automatically find your current position, in case your GPS is slow, or you dont have GPS, or you simply want the sky details of another point on the planet, you have an option to directly enter the coordinates (latitude and longitude - which you can find out from wikimapia). Combine this with the cool Time Travel feature, and you can plan when to take your girlfriend to the beach so u wont miss the sunset next week for super brownie points ;)!

Favourite must-have Android Offline Apps

work in progress!
In this must have list, I will be focusing on apps that do their work without net connectivity.
Utils
Swype apk: This is basically an alternative to the basic keyboard Input in Android. Swype greatly reduces ur typing speed by allowing u to drag ur finger over the letters of a keyboard instead of typing each letter one by one. It then guesses smartly the correct word or gives you options if more than 1 word is eligible! On some Android phones, this comes default installed, but on others you have to install it. Make sure you dont install this if you already have it installed. You wont get this on Market but you can install it from the apk link I've given.
Flashlight by Devesh Parekh apk: One touch app that turns your phone into a white full bright screen. Very useful especially if you usually keep your phone on lowest light and you one day find yourself needing some light in the dark...
Advanced Task Killerapk: EXCELLENT APP. Using this to kill unnecessary apps that run in the background, I was able to extend my Android battery by 6-8 hours (I can get max 48 hours with mild usage and no 3G).
App Monster apk: A good application manager - shows u all user apps that are installed, gives you an option to back up the apps onto your sd card (useful if you installed the app via Market and dont have apk file to install later once you uninstalled).
Cardiometer apk: Inspires you to start losing weight with all its cool features!!! Shows you your calories burnt, distance travelled, even running route on a map if GPS is enabled and so much more...
J2ME Runner Home: Run almost any Mobile Java application on your android with this.
Vignette apk: A great replacement for the default Android Camera
Chomp SMS apk: Alternative to the default Android SMS app. Has a slightly better threaded view and customizable colors.

Fun
Google Sky Map Home: THE BEST APP EVER! Detailed review at my other post here.

Games
Angry Birds apk: Amazing addictive game
Abduction apk: First bestseller on Android - a fun motion sensor game
Fish food apk: Interesting game with a new take on the old matching colours game.

For offline installation and best places to download apk files, check my previous post, Applications for Android - Online vs Offline Installation

Offline installation of Anrdoid SDK (Platforms and addons)

I installed Android SDK the other day to explore the apps platform for 2.1 (now that I have an Android phone!)…

Things you need:

A. SDK Starter package which just needs to be unzipped into a location (no installation as such):

Platform	Package	Size	MD5 Checksum
Windows	android-sdk_r07-windows.zip	23669664 bytes	69c40c2d2e408b623156934f9ae574f0
Mac OS X (intel)	android-sdk_r07-mac_x86.zip	19229546 bytes	0f330ed3ebb36786faf6dc72b8acf819
Linux (i386)	android-sdk_r07-linux_x86.tgz	17114517 bytes	e10c75da3d1aa147ddd4a5c58bfc3646

B. Platform package which is one or more of the various android versions for which you will be coding. Eg: 2.2, 2.1, 1.6, 1.5, etc.

This has to be selected and downloaded from within the Android SDK.


But Google was forcing me to download the platform package only through the SDK. Why can’t I have a set of zips I can reuse on my laptop, desltop1, desktop2 as needed instead of having to download the Android platform repeatedly from the net for each machine?


The answer was simple after a little investigation but apparently undocumented. See below steps for full offline installation:

1) First you need to download repository.xml file. To find out the download link for this file, launch Android AVD Manager (after unzipping the sdk, manually from %AndroidSDKDir%\tools\Android.bat or from the Eclipse plugin). Here, when you select available Packages, you will see the default target: http://dl.google.com/android/repository/repository.xml (maybe slightly different for you). Download this respository.xml file from the net (and close the AVD manager).

2) Open the repository.xml file and search for <sdk:platform>. You will see around 5 entries for each Android platform/version.

Identify the version you want and look for the platform filename inside the <sdk:url> tag. In my case, I wanted the 2.1 platform:

…
<sdk:platform>
     <sdk:version>2.1</sdk:version>
     <sdk:api-level>7</sdk:api-level>
     <sdk:codename/>
     <sdk:revision>02</sdk:revision>
     <sdk:min-tools-rev>6</sdk:min-tools-rev>
     <sdk:description>Android SDK Platform 2.1_r2</sdk:description>
     <sdk:desc-url>http://developer.android.com/sdk/</sdk:desc-url>
     <sdk:archives>
         <sdk:archive os="linux" arch="any">
             <sdk:size>72356359</sdk:size>
             <sdk:checksum type="sha1">a89d32ddf9d8629f15e682bdf220a88ca2392349</sdk:checksum>
             <sdk:url>android-2.1_r02-linux.zip</sdk:url>
         </sdk:archive>
         <sdk:archive os="macosx" arch="any">
             <sdk:size>71054909</sdk:size>
             <sdk:checksum type="sha1">8492e2f0b61819c4ca2e16601fc3260b341a3f06</sdk:checksum>
             <sdk:url>android-2.1_r02-macosx.zip</sdk:url>
         </sdk:archive>
         <sdk:archive os="windows" arch="any">
             <sdk:size>72950641</sdk:size>
             <sdk:checksum type="sha1">00cfb0bbbf24fc56c493990186deb0bfd44630e5</sdk:checksum>
             <sdk:url>android-2.1_r02-windows.zip</sdk:url>
         </sdk:archive>
     </sdk:archives>
 </sdk:platform>
…

3) In the url obtained from step 1 (http://dl.google.com/android/repository/repository.xml), replace repository.xml with the identified filename to get the actual url of the platform file. So in this example, the url would be: http://dl.google.com/android/repository/android-2.1_r02-windows.zip

Download this file (by typing into your browser url for eg).

4) Once the file is downloaded, extract this file into the <Androidsdk>\platforms directory. So you would end up with a new directory here as %AndroidSDKDir%\platforms\android-2.1_r02-windows.

That’s it!

If you now open the AVD manager again and click on Installed Packages, the extracted platform will show up as “SDK Platform Android a.b-update c, API d, revision e”


Use similar steps to download any Addons like Google APIs, but extract the Add-ons into %AndroidSDKDir%\add-ons directory. Just make sure that before you install the Addons, you have the corresponding platform already unzipped into the platforms directory.


To summarise, now I don’t need a net connection when to install the full android SDK – I just have to keep the following files handy:

· android-sdk_r07-windows.zip (unzip into a location where you want to install Android SDK, %AndroidSDKDir%)

· android-2.1_r02-windows.zip (unzip into %AndroidSDKDir%\platforms)

· google_apis-7_r01.zip (unzip into %AndroidSDKDir%\add-ons)

· ADT-0.9.9.zip (optional but useful Eclipse plugin – see here how to install this offline into Eclipse)



Happy droiding!

Applications for Android - Online vs Offline Installation

Here's a first hand report of everything you need to know about applications for your Android 2.1 - basically a newbies perspective!

I am going to skip a complete background of Android as it is already perfectly written over here at Wikipedia.

Applications like on any other platform can be Paid or free. But the great thing is Android has the highest paid to free ratio for any OS (mobile or PC) of 65%!!!

Ok. Whether free or paid, to get a new app on the phone, step no 1 is finding the apps.
Android Market is the official Google source with 150,000 apps and you typically find the majority listed here. Apps are also hosted on other places as listed here but I have not checked them out yet - I like to get my reviews for Android Apps from http://www.appbrain.com.


Once you've found your App, there are 2 ways to install them on your Android phone:
1) Online, i.e., directly from your Android phone by browsing Android Market. This is the best way especially if you have a cheap/affordable GPRS/3G plan on your Android phone. You can just open the MarketPlace app, browse through or search, and select to install (if its not free, you have to purchase the app first of course). Your selected app will be downloaded and installed in 1 step! You never actually have to run an exe or script or anything else to install the app.

2) Offline: If your mobile's data plan is expensive (like mine), then the only real option is to download the apps onto your PC using your affordable net connections.
For the Android OS, installation files ends with an .apk extension. Once you've downloaded the apk file your selected app, copy the apk file to your Android Phone (via wifi/usb/memorycard). Then simply use the File Browser app (present on all Android OS by default), select the apk file you copied and open it - you would then be prompted to install. Its not that complicated right?
Unfortunately, you'll find that there are very few applications being distributed officially via apk. I don't know why - especially because its not that complicated to extract an apk once you have installed it directly from Android Market.
There are some sites where popular apks are posted but you have to be cautious as it is not official (meaning it might be a tampered app or a completely different app you find after u install it - but this hasn't happened to me yet with the 25 and counting apps I installed):
http://getandroidstuff.com
http://www.freewarelovers.com/android
http://www.getjar.com


One major thorny issue I want to mention about Apps on your Android before you go downloading all those free apps. Android OS has a severe flaw that forces all apps to be installed only in the phones Memory and not in the Memory Card!!! Unbelievable but true!!!!! WHY DIDN'T ANYONE TELL ME THIS BEFORE I GOT AN ANDROID DUMB PHONE? What a major setback for a someone rooting for Android from the word go?
What this means is that although my Motorola Quench HT3 supports upto 32GB Memory card, I am stuck with installing only upto 100MB of apps supported on the Internal memory storage limit on my particular phone THANKS TO GOOGLE ##@@^%$%@@!!!!!!!!!!!!!!!!!!
This issue is finally being touched upon in the latest Android 2.2 where app developers can give the option to the user to choose where to install the apps. Unfortunately, the default is still the phone memory. And unless older apps are updated and reinstalled, it will continue to be in the phone memory. I guess I just have to finally accept that Google is becoming another money-hungry monopoly like Yahoo and Microsoft...
For me with my Android 2.1 Eclair, I have no choice but to limit my applications installations but I have not yet hit the limit mind you. But that's because I was being extra careful by keeping only apps that I use regularly. Currently, there are 20 apps I have installed on my Android.

For top must have apps for Android, check my other post, Favourite must-have Android Apps.

Motorola Quench XT3 XT502 - Day 1 Review

Finally got myself an entry level Android phone yesterday - the Motorola Quench XT3 XT502. So here's a series of quick reviews on it from an Owners point of view.

It's still not available for display in most shops but I was able to check it out the Tata Croma store at Vashi Centre Mall. They received their first HTC Quench XT3 just that morning but they agreed to let me switch it on and have a look even though it was in a brand new sealed-pack.

I was pleasantly surprised to find that unlike what is advertised on Motorola's own site, this phone came pre-installed with Android 2.1 instead of 1.6! I decided to get it right away for Rs 13691 (this includes the additional Rs700 I paid to Croma to get 2 additional years of warranty).
Also, I was able to bargain for a free 2GB Memory card.

Summary of Quench XT3 XT502
Pros
HVGA screen with 2.2 inch
Has lots of extra hardware buttons (volume +/-, camera, Dial, Cancel Call, trackball)
Cons
No FM
No Data cable
Only 262k colors
Compared to other phones, slightly lower capacity battery


Whats in the box?
It came with charger, battery, earphones. That's it.
It does not have any memory card or even data cable. Its very strange that it did not come with memory card as most of the android features including camera need SD card! But as mentioned before, after insisting on a memory card, the Croma store people gave a Sandisk 2GB microSD free.
What's worse is that the lack of data cable means it will be very frustrating for those with expensive GPRS plans and no wifi - they cannot share the internet connection being used on their PC.
The phone has only default android apps with not even a single game. The good news there are EXCELLENT free apps available. Will cover this in a later article.

Comparison with other competing phones
When I was shopping, the only real phones I could compare in this range were
Samsung Galaxy I5801 (also known as )
Song Ericsson Xperia X8
Wildfire
LG Optimus One seems to be taking forever to reach any shops here, so I couldn't consider it.

The screen size is important because most of 1000s of coolest (mostly free) Android apps out there work on the HVGA dimension of 320 x 480 pixels. There is a chance that if your Android phone has a different dimension, apps may run incorrectly if not at all. So this was my biggest priority.
Out of the available phones, no other phone provides HVGA and with 2.2 inch at this price range:
I5801: Only WQVGA although it has 2.2inch size.
Xperia X8: It has HVGA but with very small 2 inch size.

Here is a detailed comparison with some more competing phones:

Model	Sony Ericsson Xperia X8	Sony Ericsson Xperia X10 Mini	Samsung Galaxy 3	HTC Wildfire	HTC Tattoo A3232	Motorola Quench XT3 XT502	LG Optimus One
Cost@Flipkart.com	Rs.13950.00	Rs.12899.00	Rs.11899.00	Rs.15079.00	Rs.12999.00	Rs.12449.00	NA
Specialoffers	None	None	Free Reebok Watch worth Rs2599	Free HTC Leather Pouch worth Rs.349	None	Free Adidas Jacket & 4 Movie Tickets of across multiplexes worth Rs.3599	NA
CPU	600Mhz	QualcommMSM 7227 600MHz	Qualcomm(QSC6270), 667 MHz	QualcommMSM 7225 528 MHz	QualcommMSM 7225 528 MHz	QualcommMSM 7227 600MHz	600MHz processor
Memory	128MB?	128MB?	130MB?	384 MBRAM; 512 MB ROM	256 MB RAM, 512 MB ROM	100 MB storage, 256 MB RAM; 512 MB ROM	170 MB
A-GPS	Y	Y	Y	Y	Y	Y	Y
Wifi	Y	Y	Y	Y	Y	Y	Y
Camera	3.15MP	5MPAuto-focus?	3.2MP?Auto-focus?	5MPAuto-focus?	3.15MPFixed-focus?	3.15MP fixed focus + Flash	3.15MP Auto-focus?
Resolution	320 x480 pixels (HVGA)	240 x320 pixels (QVGA)	240 x400 (WQVGA) TFT	240 x320 (QVGA) TFT	240 x320 (QVGA) TFT	320 x480 pixels (HVGA)	320 x480 pixels (HVGA) TFT
Size"	3	2.55	3.2	3.2	2.8	3.2	3.2
Colors	16M	16M	16M	16M	65k	256k	256k
Touch	Capacitative	Capacitative	Capacitative	Capacitative	Resistive	Capacitative	Capacitative
Dimensions	99.0 x54.0 x 15.0 mm	83.0 x50.0 x 16.0mm	113.5x 55 x 12.55 mm	106.8x 60.4 x 12 mm	106 x55.2 x 14 mm	114.9x 56.8 x 12.6 mm	113.5x 59 x 13.3 mm
Weight	104g	88g	?	118 g	113 g	114 g	129 g
OS	Android1.6	Android1.6	Android2.1	Android2.1	Android1.6?	Android2.1	Android2.2
ExtMemory	MicroSDupto 16MB, 2GB included	MicroSDupto 16MB, 2 GB included	MicroSDupto 32MB, 2 GB included	MicroSDupto 32MB, 2 GB included	MicroSD	microSD,up to 32GB. None included	microSD,up to 32GB, 2GB included
Battery	Li-Po1200 mAh	Li-Po950 mAh	Li-ion,1,500 mAh	Li-Ion,1300mAh	1100mAh?	Li-Po1270 mAh	Li-Ion1500 mAh
FMRadio	Y	Y	?	Y	Y	N?	Y
3.5mmaudio	Y	Y	Y	Y	Y	Y	Y
3G	Y	Y	Y	Y	Y	Y	Y

First time Android user
Since I haven't used Android before, I decided to follow the manual exactly:
Open the cover and remove battery (not really necessary to remove battery for inserting sim and memory, but I just wanted to follow the instructions!).
Insert Sim and my 8GB Memory card from my Samsung star phone (which died last week after a water attack!)
Replace battery.
Charge for 3 hours!

Finally switched on the phone at 12am. First thing I did was a factory reset.

WARNING: Factory Reset will clear all the data on the phone like Contacts, Call logs, installed applications and their data, etc.
But don't worry if you are doing this on a new phone - factory Reset does not affect any of your data on the memory card nor the data on the Sim.

Performing a Factory Reset makes sure that I got a fresh Android OS with no overheads.
For Factory Reset, go to

After doing the Factory Reset, the phone restarts. After a few minutes, it displays the Welcome message with a Tour which you can skip. Then, it prompts you to enter your Google account details or create a Google account. But I skipped this step as it needs net connectivity and my GPRS plan is very expensive. This step will be prompted when you click on any of the Google apps on the phone like GMail, Lattitude, etc.

Pre-installed Apps
The Gallery app is where all your media is displayed. When I opened this app for the first time, it showed as busy with a message "Loading". I guess it was finding all the 3Gb of pics and videos on my old Memory Card. I switched to another app and when I came after 2 mins, it allowed me to view whatever media was loaded while it continued finding more.
But it was REALLY SMOOTH after that! The images are beautifully stacked in the folders. And when clicking on the images loaded them instantly -scrolling to the next pics was just as smooth with instant loading. It think it buffers the next few pics because when I started scrolling through the pics in full screen very fast, my 3MP resolution pics was pixellated and it takes another 1 second for the image to become fully clear.

Anyway, the good part is that it is extremley smooth and fast after that first search. I was interesting that it found each and every single image in my memory card from all scattered locations.

Maybe it's because I have never owned a multi-touch phone before - but I was downright AMAZED by the pinch and unpinch to Zoom and unzoom pics. I mean not only is it very intuituve but it was SUPER FAST and SMOOTH!!!

Another thing is that it showed as busy for about 2 minutes while loading the 2GB pics/videos in my old 8GB Memory Card data before it allowed to start looking at to load all the 3GB of pics n videos in the memory card.

The usual Address Book feature is provided via the Contacts app. However, none of the Phone numbers in the Sim are available automatically in the Contacts . Thankfully, there is an option to Import the phone numbers from your Sim - you can select Import All or just choose the numbers you want to import. To import all the 100 odd contacts in my sim, it took about 3 minutes. Like with the one-time slow processes, I simply switched to another task as the Android can Multitask unlike the iPhone you know :)!

Battery
Battery lasts for 1 day with average gaming and calls with no wifi/bluetooth. This was disappointing for me coming from my Samsung Star S5233A which could last me 2 days easily and sometimes even 4. But then this was expected because Android of course supports multi tasking. So you can have your music player runnng, taking cam pics, use your browser, and create a document without closing any of the apps. But since these apps have no close option, they would continue to run in the background for a while before being closed costing precious processor cycles which in turn means more battery usage.
To resolve this, there is an app called Advanced Task Killer - this shows all running apps and u can kill the apps from here too. Also, there is a 1 click shortcut which remembers the apps u selected and killed earlier and automatically kills them! So what I do is just click on this shortcut when I am done using the phone. I was able to get 1.5 to 2 days of battery life once I started doing this.

Tips while building your own computer

Was doing some research for building a computer – was real rusty with the latest info. Summarised my learnings below:

 

SMPS

·         An SMPS cannot give the complete power that it is rated for. The actual power provided varies with temperature, load on the system, etc.

So Corsair VX450W (which is an excellent SMPS) can only give 382.5W at 50% load (that is 85% efficiency. Refer here).

So always buy an SMPS higher than your need.

·         Calculate your expected power need of your computer using this online calculator.

So always buy an SMPS higher than your need.

 

Hard Disks

·         For faster speeds, go for a 7200rpm HDD instead of 5400rpm.

·         A HDD with higher cache (32MB is really good) improves real-life file copy speeds dramatically (especially when going for a slower rpm HDD).

·         Theoretical Interface Speeds:

SATA 3.0 (6Gbps/600Mbps) > USB 3.0 (5Gbps) > SATA 2.0 (3.0Gbps) > SATA 1.0 (1.5Gbps/150 MBps) > USB 2.0 (60MBps)

·         eSATA and eSATAP have the same speeds as the respective SATA versions.

·         eSATA devices need additional power via a separate power cable. USB and eSATAP have built in power through the cable

·         Unlike USB 2, USB 3.0 is bidirectional like all the SATA versions.

·         All USB interfaces (including 3.0) need some bridging to communicate to the HDD ports on the MB which definitely slows down actual data throughput (refer here)

·         For current magnetic HDD transfer rate of 240Mbit/s (average), the latest SATA3.0 or USB 3 interface is overkill. But the latest solidstate drives push the boundary to 355 (refer here) and can benefit from SATA III…

·         If you have 2 drives, try to setup a Raid 1+0 configuration. This gives you much faster speeds and also disk failure tolerance. But the cost is that less space will be available. See how here

 

Dangerous scripts - safety tips

Hadn't checked Orkut for a while - and was shocked to yesterday to see a virulent script-worm spreading across it... even some of my very net-savvy friends were victims too.

If you clicked on this script link while logged in to Orkut, it steals your password, enrolls u automatically in strange communities, and makes auto-spam posts onto all your friends testimonials, communities, etc. Read details here: http://www.google.co.in/support/forum/p/orkut/thread?tid=0cbd83ba084be690&hl=en


Bottlomline - don't get on the net if you can't learn these simple safety tips:
1) Don't blindly click on Yes when getting browser prompts to run suspicious scripts.
2) Don't ever click on suspicious links or emails even if it's from someone you know well.
3) Don't ever ever click on suspicious links or emails while you are logged on into an online account (specially popular ones like Yahoo, gmail, orkut, facebook, etc)

Also, have at least one good Firewall software running (even though XP onwards have one built-in). Even a free one like Commodo if you are advanced enough.

UK Postcode regular expression

A friend of mine asked for a regular expression for UK Post codes… got so many confusing results on the net that I decided to make it myself.

First had to find the rules for UK Postcode. A quick search got me a UK government site, (old site was: http://www.cabinetoffice.gov.uk/govtalk/schemasstandards/e-gif/datastandards/address/postcode.aspx as of October 2009. Updated to new site) http://interim.cabinetoffice.gov.uk/govtalk/schemasstandards/e-gif/datastandards/address/postcode.aspx, as of July 2011, which shows the rules as follows:

Permitted Format	Example Postcode
AN NAA	M1 1AA
ANN NAA	M60 1NW
AAN NAA	CR2 6XH
AANN NAA	DN55 1PT
ANA NAA	W1A 1HQ
AANA NAA	EC1A 1BB

Also

* The letters Q, V and X are not used in the first position.

* The letters I, J and Z are not used in the second position.

* The only letters to appear in the third position are A, B, C, D, E, F, G, H, J, K, S, T, U and W.

* The only letters to appear in the fourth position are A, B, E, H, M, N, P, R, V, W, X and Y.

* The second half of the Postcode is always consistent numeric, alpha, alpha format and the letters C, I, K, M, O and V are never used.

* GIR 0AA is a Postcode that was issued historically and does not confirm to current rules on valid Postcode formats, It is however, still in use.

Was able to come up with this basic Regular expression that does UK Postcode validation:

^([A-PR-UWYZ](([0-9](([0-9]|[A-HJKSTUW])?)?)|([A-HK-Y][0-9]([0-9]|[ABEHMNPRVWXY])?)) [0-9][ABD-HJLNP-UW-Z]{2})|GIR 0AA$

This will validate 100% as per the assumed rules above.
Note however that it is certainly not optimized… I couldn't find any online regular expression optimizer and I’ll have to become a regex expert to do anything about it. Maybe I can take automatic regex optmizer thing as a mini-project. But for now, the unoptimized version will have to do...

Here is the regular expression in action inside Java code:

public static void validate(String code) {

String regexp="^([A-PR-UWYZ](([0-9](([0-9]|[A-HJKSTUW])?)?)|([A-HK-Y][0-9]([0-9]|[ABEHMNPRVWXY])?)) [0-9][ABD-HJLNP-UW-Z]{2})|GIR 0AA$";

Pattern pattern = Pattern.compile(regexp);

Matcher matcher = pattern.matcher(code.toUpperCase());

if (matcher.matches()) {

System.out.println("This is a valid UK Postcode.");

} else {

System.out.println("This is not a valid UK Postcode.");

}

}

In a real life scenario, you may need to convert the received code into upper case before calling validate to be safe. Or better yet, have upper and lower case validation inside the regular expression itself! Plus of course the optimization…

Support for NTLMv2 with Apache HttpClient using JCIFS

Here, I'll explain how to get NTLMv2 support in HttpClient 3.x using JCIFS by using 1 addditional class and inserting 1 new line into your existing calls to HttpClient.
Of course you can use JCIFS NTLM authentication directly in Java even if you don't have Apache HttpClient - I'm not looking at that but you can refer to this guide on JCIFS home page.

HttpClient

Apache's HttpClient provides some useful encapsulation for fetching/posting data over HTTP through Java code. The common version is HttpClient 3.x whereas the latest version is HttpClient 4.x.

NTLM

One of HttpClient's advantages is that it has built in support to manage communications over an NTLM proxy.

NTLM is a very closely guarded Proxy protocol used by Microsoft but still popularly used.

JCIFS

Unfortunately, HttpClient does not have built in support for NTLM v2. The good news is, it allows you to integrate NTLMv2 support through another library called JCIFS.

JCIFS is an Open Source  client library that implements the CIFS/SMB networking protocol in 100% Java. See more details/download from here. But since JCIFS started NTLMv2 support only from 1.3.0, make sure you have the latest JCIFS jar (I tested with JCIFS 1.3.14).

JCIFS in HttpClient

Thankfully, HttpClient 4.x home site has a page containing unofficial steps for integrating JCIFS into HTTPClient. However, these steps will only work for 4.x and not for for 3.x. This is because HttpClient 4.x is not backward compatible with 3.x. There are major changes like package structures, new Engines instead of States, etc.

Since I was using HttpClient 3.x, I started thinking of upgrading to 4.x – but it was obvious upgrading to 4.x from 3.x was a nightmare just to get NTLMv2 support.

Thankfully, it was just a matter of understanding HttpClient 3.x internal calls from the source. I was able to create 1 simple class that will do the integration in 1 smooth move.

Steps and source-code

1)      I created a new class, JCIFS_NTLMScheme.java that would be used in place of HTTPClient's NTLMScheme.

This new class simply makes calls to JCIFS internally to generate NTLMv2's Type1, 2 and 3 messages- just reimplemented the methods with slight logic changes to generate the messages:

package org.xyz;

import java.io.IOException;

import org.apache.commons.httpclient.Credentials;

import org.apache.commons.httpclient.HttpMethod;

import org.apache.commons.httpclient.NTCredentials;

import org.apache.commons.httpclient.auth.AuthChallengeParser;

import org.apache.commons.httpclient.auth.AuthScheme;

import org.apache.commons.httpclient.auth.AuthenticationException;

import org.apache.commons.httpclient.auth.InvalidCredentialsException;

import org.apache.commons.httpclient.auth.MalformedChallengeException;

import org.sac.crosspather.common.util.AppLogger;

import org.sac.crosspather.common.util.HTTPHelper;

/**

 * This is a reimplementation of HTTPClient 3.x's

 * org.apache.commons.httpclient.auth.NTLMScheme.<BR/>

 * It will basically use JCIFS (v1.3.15) in order to provide added support for

 * NTLMv2 (instead of trying to create its own Type, 2 and 3 messages). <BR/>

 * This class has to be registered manually with HTTPClient before setting

 * NTCredentials: AuthPolicy.registerAuthScheme(AuthPolicy.NTLM,

 * JCIFS_NTLMScheme.class); <BR/>

 * Will <B>not</B> work with HttpClient 4.x which requires AuthEngine to be overriden instead of AuthScheme.

 *

 * @author Sachin M

 */

public class JCIFS_NTLMScheme implements AuthScheme {

       private static AppLogger logger = new AppLogger(HTTPHelper.class.getName());

       /** NTLM challenge string. */

       private String ntlmchallenge = null;

       private static final int UNINITIATED = 0;

       private static final int INITIATED = 1;

       private static final int TYPE1_MSG_GENERATED = 2;

       private static final int TYPE2_MSG_RECEIVED = 3;

       private static final int TYPE3_MSG_GENERATED = 4;

       private static final int FAILED = Integer.MAX_VALUE;

       /** Authentication process state */

       private int state;

       public JCIFS_NTLMScheme() throws AuthenticationException {

              // Check if JCIFS is present. If not present, do not proceed.

              try {

                     Class.forName("jcifs.ntlmssp.NtlmMessage",false,this.getClass().getClassLoader());

              } catch (ClassNotFoundException e) {

                     throw new AuthenticationException("Unable to proceed as JCIFS library is not found.");

              }

       }

      

       public String authenticate(Credentials credentials, HttpMethod method)

                     throws AuthenticationException {

              logger.doLog(AppLogger.FINEST,

                           "Enter JCIFS_NTLMScheme.authenticate(Credentials, HttpMethod)",

                           null);

              if (this.state == UNINITIATED) {

                     throw new IllegalStateException(

                                  "NTLM authentication process has not been initiated");

              }

              NTCredentials ntcredentials = null;

              try {

                     ntcredentials = (NTCredentials) credentials;

              } catch (ClassCastException e) {

                     throw new InvalidCredentialsException(

                                  "Credentials cannot be used for NTLM authentication: "

                                                + credentials.getClass().getName());

              }

             

              NTLM ntlm = new NTLM();

              ntlm.setCredentialCharset(method.getParams().getCredentialCharset());

              String response = null;

              if (this.state == INITIATED || this.state == FAILED) {

                     response = ntlm.generateType1Msg(ntcredentials.getHost(),

                                  ntcredentials.getDomain());

                     this.state = TYPE1_MSG_GENERATED;

              } else {

                     response = ntlm.generateType3Msg(ntcredentials.getUserName(),

                                  ntcredentials.getPassword(), ntcredentials.getHost(),

                                  ntcredentials.getDomain(), this.ntlmchallenge);

                     this.state = TYPE3_MSG_GENERATED;

              }

              return "NTLM " + response;

       }

       public String authenticate(Credentials credentials, String method,

                     String uri) throws AuthenticationException {

              throw new RuntimeException(

                           "Not implemented as it is deprecated anyway in Httpclient 3.x");

       }

       public String getID() {

              throw new RuntimeException(

                           "Not implemented as it is deprecated anyway in Httpclient 3.x");

       }

       /**

        * Returns the authentication parameter with the given name, if available.

        *

        * <p>

        * There are no valid parameters for NTLM authentication so this method

        * always returns <tt>null</tt>.

        * </p>

        *

        * @param name

        *            The name of the parameter to be returned

        *

        * @return the parameter with the given name

        */

       public String getParameter(String name) {

              if (name == null) {

                     throw new IllegalArgumentException("Parameter name may not be null");

              }

              return null;

       }

       /**

        * The concept of an authentication realm is not supported by the NTLM

        * authentication scheme. Always returns <code>null</code>.

        *

        * @return <code>null</code>

        */

       public String getRealm() {

              return null;

       }

       /**

        * Returns textual designation of the NTLM authentication scheme.

        *

        * @return <code>ntlm</code>

        */

       public String getSchemeName() {

              return "ntlm";

       }

       /**

        * Tests if the NTLM authentication process has been completed.

        *

        * @return <tt>true</tt> if Basic authorization has been processed,

        *         <tt>false</tt> otherwise.

        *

        * @since 3.0

        */

       public boolean isComplete() {

              return this.state == TYPE3_MSG_GENERATED || this.state == FAILED;

       }

       /**

        * Returns <tt>true</tt>. NTLM authentication scheme is connection based.

        *

        * @return <tt>true</tt>.

        *

        * @since 3.0

        */

       public boolean isConnectionBased() {

              return true;

       }

       /**

        * Processes the NTLM challenge.

        *

        * @param challenge

        *            the challenge string

        *

        * @throws MalformedChallengeException

        *             is thrown if the authentication challenge is malformed

        *

        * @since 3.0

        */

       public void processChallenge(final String challenge)

                     throws MalformedChallengeException {

              String s = AuthChallengeParser.extractScheme(challenge);

              if (!s.equalsIgnoreCase(getSchemeName())) {

                     throw new MalformedChallengeException("Invalid NTLM challenge: "

                                  + challenge);

              }

              int i = challenge.indexOf(' ');

              if (i != -1) {

                     s = challenge.substring(i, challenge.length());

                     this.ntlmchallenge = s.trim();

                     this.state = TYPE2_MSG_RECEIVED;

              } else {

                     this.ntlmchallenge = "";

                     if (this.state == UNINITIATED) {

                           this.state = INITIATED;

                     } else {

                           this.state = FAILED;

                     }

              }

       }

       private class NTLM {

           /** Character encoding */

           public static final String DEFAULT_CHARSET = "ASCII";

          

           /**

               * The character was used by 3.x's NTLM to encode the username and

               * password. Apparently, this is not needed in when passing username,

               * password from NTCredentials to the JCIFS library

               */

           private String credentialCharset = DEFAULT_CHARSET;

          

              void setCredentialCharset(String credentialCharset) {

                     this.credentialCharset = credentialCharset;

              }

              private String generateType1Msg(String host, String domain) {

                     jcifs.ntlmssp.Type1Message t1m = new jcifs.ntlmssp.Type1Message(jcifs.ntlmssp.Type1Message.getDefaultFlags(),

                                  domain, host);

                     return jcifs.util.Base64.encode(t1m.toByteArray());

              }

              private String generateType3Msg(String username, String password, String host,

                           String domain, String challenge) {

                     jcifs.ntlmssp.Type2Message t2m;

                     try {

                           t2m = new jcifs.ntlmssp.Type2Message(jcifs.util.Base64.decode(challenge));

                     } catch (IOException e) {

                           throw new RuntimeException("Invalid Type2 message", e);

                     }

                     jcifs.ntlmssp.Type3Message t3m = new jcifs.ntlmssp.Type3Message(t2m, password, domain,

                                  username, host, 0);

                     return jcifs.util.Base64.encode(t3m.toByteArray());

              }

       }

}

2)      Then it was Register the new JCIFS_NTLMScheme class as the replacement for NTLMScheme by using the following command:

AuthPolicy.registerAuthScheme(AuthPolicy.NTLM, org.xyz.JCIFS_NTLMScheme.class);

 (AuthPolicy is a class in HTTPClient 3.x jar)

That's it! Use your HTTPClient as normal but just make sure you call the above register command before you create and bind the HttpClient's NTCredentials class