Wednesday, May 12, 2010

Support for NTLMv2 with Apache HttpClient using JCIFS

Here, I'll explain how to get NTLMv2 support in HttpClient 3.x using JCIFS by using 1 addditional class and inserting 1 new line into your existing calls to HttpClient.
Of course you can use JCIFS NTLM authentication directly in Java even if you don't have Apache HttpClient - I'm not looking at that but you can refer to this guide on JCIFS home page.

HttpClient
Apache's HttpClient provides some useful encapsulation for fetching/posting data over HTTP through Java code. The common version is HttpClient 3.x whereas the latest version is HttpClient 4.x.

NTLM
One of HttpClient's advantages is that it has built in support to manage communications over an NTLM proxy.
NTLM is a very closely guarded Proxy protocol used by Microsoft but still popularly used.

JCIFS
Unfortunately, HttpClient does not have built in support for NTLM v2. The good news is, it allows you to integrate NTLMv2 support through another library called JCIFS.
JCIFS is an Open Source  client library that implements the CIFS/SMB networking protocol in 100% Java. See more details/download from here. But since JCIFS started NTLMv2 support only from 1.3.0, make sure you have the latest JCIFS jar (I tested with JCIFS 1.3.14).

JCIFS in HttpClient
Thankfully, HttpClient 4.x home site has a page containing unofficial steps for integrating JCIFS into HTTPClient. However, these steps will only work for 4.x and not for for 3.x. This is because HttpClient 4.x is not backward compatible with 3.x. There are major changes like package structures, new Engines instead of States, etc.

Since I was using HttpClient 3.x, I started thinking of upgrading to 4.x – but it was obvious upgrading to 4.x from 3.x was a nightmare just to get NTLMv2 support.

Thankfully, it was just a matter of understanding HttpClient 3.x internal calls from the source. I was able to create 1 simple class that will do the integration in 1 smooth move.

Steps and source-code
1)      I created a new class, JCIFS_NTLMScheme.java that would be used in place of HTTPClient's NTLMScheme.
This new class simply makes calls to JCIFS internally to generate NTLMv2's Type1, 2 and 3 messages- just reimplemented the methods with slight logic changes to generate the messages:
package org.xyz;

import java.io.IOException;

import org.apache.commons.httpclient.Credentials;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.NTCredentials;
import org.apache.commons.httpclient.auth.AuthChallengeParser;
import org.apache.commons.httpclient.auth.AuthScheme;
import org.apache.commons.httpclient.auth.AuthenticationException;
import org.apache.commons.httpclient.auth.InvalidCredentialsException;
import org.apache.commons.httpclient.auth.MalformedChallengeException;
import org.sac.crosspather.common.util.AppLogger;
import org.sac.crosspather.common.util.HTTPHelper;

/**
 * This is a reimplementation of HTTPClient 3.x's
 * org.apache.commons.httpclient.auth.NTLMScheme.<BR/>
 * It will basically use JCIFS (v1.3.15) in order to provide added support for
 * NTLMv2 (instead of trying to create its own Type, 2 and 3 messages). <BR/>
 * This class has to be registered manually with HTTPClient before setting
 * NTCredentials: AuthPolicy.registerAuthScheme(AuthPolicy.NTLM,
 * JCIFS_NTLMScheme.class); <BR/>
 * Will <B>not</B> work with HttpClient 4.x which requires AuthEngine to be overriden instead of AuthScheme.
 *
 * @author Sachin M
 */
public class JCIFS_NTLMScheme implements AuthScheme {

       private static AppLogger logger = new AppLogger(HTTPHelper.class.getName());

       /** NTLM challenge string. */
       private String ntlmchallenge = null;

       private static final int UNINITIATED = 0;
       private static final int INITIATED = 1;
       private static final int TYPE1_MSG_GENERATED = 2;
       private static final int TYPE2_MSG_RECEIVED = 3;
       private static final int TYPE3_MSG_GENERATED = 4;
       private static final int FAILED = Integer.MAX_VALUE;

       /** Authentication process state */
       private int state;

       public JCIFS_NTLMScheme() throws AuthenticationException {
              // Check if JCIFS is present. If not present, do not proceed.
              try {
                     Class.forName("jcifs.ntlmssp.NtlmMessage",false,this.getClass().getClassLoader());
              } catch (ClassNotFoundException e) {
                     throw new AuthenticationException("Unable to proceed as JCIFS library is not found.");
              }
       }
      
       public String authenticate(Credentials credentials, HttpMethod method)
                     throws AuthenticationException {
              logger.doLog(AppLogger.FINEST,
                           "Enter JCIFS_NTLMScheme.authenticate(Credentials, HttpMethod)",
                           null);

              if (this.state == UNINITIATED) {
                     throw new IllegalStateException(
                                  "NTLM authentication process has not been initiated");
              }

              NTCredentials ntcredentials = null;
              try {
                     ntcredentials = (NTCredentials) credentials;
              } catch (ClassCastException e) {
                     throw new InvalidCredentialsException(
                                  "Credentials cannot be used for NTLM authentication: "
                                                + credentials.getClass().getName());
              }
             
              NTLM ntlm = new NTLM();
              ntlm.setCredentialCharset(method.getParams().getCredentialCharset());
              String response = null;
              if (this.state == INITIATED || this.state == FAILED) {
                     response = ntlm.generateType1Msg(ntcredentials.getHost(),
                                  ntcredentials.getDomain());
                     this.state = TYPE1_MSG_GENERATED;
              } else {
                     response = ntlm.generateType3Msg(ntcredentials.getUserName(),
                                  ntcredentials.getPassword(), ntcredentials.getHost(),
                                  ntcredentials.getDomain(), this.ntlmchallenge);
                     this.state = TYPE3_MSG_GENERATED;
              }
              return "NTLM " + response;

       }

       public String authenticate(Credentials credentials, String method,
                     String uri) throws AuthenticationException {
              throw new RuntimeException(
                           "Not implemented as it is deprecated anyway in Httpclient 3.x");
       }

       public String getID() {
              throw new RuntimeException(
                           "Not implemented as it is deprecated anyway in Httpclient 3.x");
       }

       /**
        * Returns the authentication parameter with the given name, if available.
        *
        * <p>
        * There are no valid parameters for NTLM authentication so this method
        * always returns <tt>null</tt>.
        * </p>
        *
        * @param name
        *            The name of the parameter to be returned
        *
        * @return the parameter with the given name
        */
       public String getParameter(String name) {
              if (name == null) {
                     throw new IllegalArgumentException("Parameter name may not be null");
              }
              return null;
       }

       /**
        * The concept of an authentication realm is not supported by the NTLM
        * authentication scheme. Always returns <code>null</code>.
        *
        * @return <code>null</code>
        */
       public String getRealm() {
              return null;
       }

       /**
        * Returns textual designation of the NTLM authentication scheme.
        *
        * @return <code>ntlm</code>
        */
       public String getSchemeName() {
              return "ntlm";
       }

       /**
        * Tests if the NTLM authentication process has been completed.
        *
        * @return <tt>true</tt> if Basic authorization has been processed,
        *         <tt>false</tt> otherwise.
        *
        * @since 3.0
        */
       public boolean isComplete() {
              return this.state == TYPE3_MSG_GENERATED || this.state == FAILED;
       }

       /**
        * Returns <tt>true</tt>. NTLM authentication scheme is connection based.
        *
        * @return <tt>true</tt>.
        *
        * @since 3.0
        */
       public boolean isConnectionBased() {
              return true;
       }

       /**
        * Processes the NTLM challenge.
        *
        * @param challenge
        *            the challenge string
        *
        * @throws MalformedChallengeException
        *             is thrown if the authentication challenge is malformed
        *
        * @since 3.0
        */
       public void processChallenge(final String challenge)
                     throws MalformedChallengeException {
              String s = AuthChallengeParser.extractScheme(challenge);
              if (!s.equalsIgnoreCase(getSchemeName())) {
                     throw new MalformedChallengeException("Invalid NTLM challenge: "
                                  + challenge);
              }
              int i = challenge.indexOf(' ');
              if (i != -1) {
                     s = challenge.substring(i, challenge.length());
                     this.ntlmchallenge = s.trim();
                     this.state = TYPE2_MSG_RECEIVED;
              } else {
                     this.ntlmchallenge = "";
                     if (this.state == UNINITIATED) {
                           this.state = INITIATED;
                     } else {
                           this.state = FAILED;
                     }
              }
       }

       private class NTLM {
           /** Character encoding */
           public static final String DEFAULT_CHARSET = "ASCII";
          
           /**
               * The character was used by 3.x's NTLM to encode the username and
               * password. Apparently, this is not needed in when passing username,
               * password from NTCredentials to the JCIFS library
               */
           private String credentialCharset = DEFAULT_CHARSET;
          
              void setCredentialCharset(String credentialCharset) {
                     this.credentialCharset = credentialCharset;
              }

              private String generateType1Msg(String host, String domain) {
                     jcifs.ntlmssp.Type1Message t1m = new jcifs.ntlmssp.Type1Message(jcifs.ntlmssp.Type1Message.getDefaultFlags(),
                                  domain, host);
                     return jcifs.util.Base64.encode(t1m.toByteArray());
              }

              private String generateType3Msg(String username, String password, String host,
                           String domain, String challenge) {
                     jcifs.ntlmssp.Type2Message t2m;
                     try {
                           t2m = new jcifs.ntlmssp.Type2Message(jcifs.util.Base64.decode(challenge));
                     } catch (IOException e) {
                           throw new RuntimeException("Invalid Type2 message", e);
                     }

                     jcifs.ntlmssp.Type3Message t3m = new jcifs.ntlmssp.Type3Message(t2m, password, domain,
                                  username, host, 0);
                     return jcifs.util.Base64.encode(t3m.toByteArray());
              }
       }
}

2)      Then it was Register the new JCIFS_NTLMScheme class as the replacement for NTLMScheme by using the following command:
AuthPolicy.registerAuthScheme(AuthPolicy.NTLM, org.xyz.JCIFS_NTLMScheme.class);
 (AuthPolicy is a class in HTTPClient 3.x jar)

That's it! Use your HTTPClient as normal but just make sure you call the above register command before you create and bind the HttpClient's NTCredentials class

42 comments:

  1. Hello and thanks for this useful post. I have implemented the code above using HTTPClient 3 and JCIFS, but I am getting a 500 HTTP error and a "The function requested is not supported" error msg from the server. Any ideas?

    ReplyDelete
  2. Hey Andy. I never came across this error but I have some suggestions...
    Guess you know that the HTTP error code as 500, stands for internal server error. In such a case, you should be able to get a full stack trace of the exception from the server too... If the error is on a browser like Internet Explorer, you can simply go to the advanced tab and uncheck the "Show friendly error" option - you should get the stack displayed in the browser next time you attempt the connection.
    If you aren't able to get a clue from the stack, post the stack back here.
    Also, could you show the code where you create NTCredentials and how you use them (star out any usernames and passwords of course :) )?
    I have some more ideas if that doesnt work either...

    ReplyDelete
  3. @Andy
    I got a similar error. The reason was that the LAN Manager Authentication Level in the Local Security Policy of the win2k machine that I was working on was set to "Send NTLMV2 response only, refuse LM and NTLM.."
    Also all the flags were enabled in the setting "Minimum session security for NTLM SSP ......"

    If you relax the settings of LAN Manager Authentication level and disable the NTLM V2 flag in the session security settings things should work fine

    ReplyDelete
  4. Naveen, are you using JCIFS or just Apache HTTPClient? Good to see that turning off NTLM v2 works for you but the most common reason for using JCIFS was to make your software compatible with NTLM v2.

    NTLM v1 is already natively supported in Apache HTTPClient...

    ReplyDelete
  5. Sachin,

    I have a customer requirement in which they have mandated NTLMV2 only. The local policy settings on their Win2k server is

    1. Send NTLMV2 response only, refuse LM and NTLM..
    2. Also all the flags were enabled in the setting "Minimum session security for NTLM SSP ......clients"
    3. Also all the flags were enabled in the setting "Minimum session security for NTLM SSP ......servers"

    I tried out your code by creating a JCIFS_NTLMScheme class and registering it with Httpclient 3.x. as mentioned in your example above. I still get the error "The function requested is not supported".

    Can you please tell me if I am missing something here. Does JCIFS work with the above security setting?

    Thanks in advance

    ReplyDelete
  6. Sachin,
    Sorry,I forgot to mention that I am using only HttpClient 3.x. I used the jcifs-1.1.11.jar only for getting the JCIFS_NTLMScheme.java to compile and run.

    ReplyDelete
  7. Hi Naveen

    First of all, I didn't really have access to my Server details - so I have no clue about the policy settings. However, JCIFS is supposed to have full NTLMv2 support...

    I think your problem is elsewhere - NTLMv2 support in JCIFS started only from JCIFS version 1.3.0 whereas it looks like you are using version 1.1.11. Make sure you have the latest JCIFS (I had done my testing with 1.3.14 - it worked flawlessly with NTLMv2 and I know my proxy server has full security on).

    Another issue is that it you said you are using the JCIFS jar file only for compiling JCIFS_NTLMScheme.java. However, you have to ensure that the JCIFS jar file is available in the classpath at runtime as well.

    Again, try to get the stack trace from any logs.

    Good luck.

    ReplyDelete
    Replies
    1. I was getting 401: Unauthorised error so have been searching for solution and found your post. I implemented the code as per what is stated in the post but I still get 401:Unauthorised error. Any clue where to look?

      Delete
  8. awesome,
    thanks & keep up the good work.

    ReplyDelete
  9. I am not able to compile code with jcifs-1.3.16.zip. Do I need any other library? BTW, can I implement this solution in JDK1.4?

    ReplyDelete
  10. Never mind. It is compiled in JDK 1.4.But, it is not working with apache axis2:


    AuthPolicy.registerAuthScheme(AuthPolicy.NTLM, projectp.pic.ui.server.JCIFS_NTLMScheme.class);

    HttpTransportProperties.Authenticator auth = new HttpTransportProperties.Authenticator();
    auth.setUsername(username);
    auth.setPassword(password);
    auth.setDomain(domain);
    auth.setHost(host);
    auth.setPort(80);

    List authPrefs = new ArrayList(1);
    authPrefs.add(AuthPolicy.NTLM);
    auth.setAuthSchemes(authPrefs);
    String targetPoint = "http://"+host+"/_layouts/PPUpload.asmx";


    UploadStub lists = new UploadStub(targetPoint);
    lists._getServiceClient().getOptions().setProperty(org.apache.axis2.transport.http.HTTPConstants.AUTHENTICATE, auth);


    Any idea?

    ReplyDelete
  11. Hi All,

    Shall we use it on AXIS 1.4 ? Because in our scenario we are using Axis1.4 with HTTPClient3.1 to call .Net WCF Web Service. So we have faced 401(Unauthorized)error while NTLM V2 Authentication.

    We have tried to used above JCIFS_NTLMScheme class and following code for Stub Calling but 401 error was thrown.


    org.apache.axis.client.Call _call = createCall();
    _call.setOperation(_operations[2]);
    _call.setUseSOAPAction(true);
    _call.setSOAPActionURI("test.pkg/EngineService/Login");
    _call.setEncodingStyle(null);
    _call.setProperty(org.apache.axis.client.Call.SEND_TYPE_ATTR, Boolean.FALSE);
    _call.setProperty(org.apache.axis.AxisEngine.PROP_DOMULTIREFS, Boolean.FALSE);
    _call.setSOAPVersion(org.apache.axis.soap.SOAPConstants.SOAP11_CONSTANTS);
    _call.setOperationName(new javax.xml.namespace.QName("", "Login"));
    AuthPolicy.registerAuthScheme(AuthPolicy.NTLM, test.pkg.JCIFS_NTLMScheme.class);
    setRequestHeaders(_call);
    setAttachments(_call);
    java.lang.Object _resp = _call.invoke(new java.lang.Object[] {loginId, password, authMode});

    Please suggest for this issue.

    ReplyDelete
    Replies
    1. Hi Debasish,

      I have the same requirement to use it in Axis 1.4. Did you had it worked in 1.4?

      Thanks,
      Justin.

      Delete
  12. anyone had any issues with this on jdk 1.5?

    ReplyDelete
  13. This program i cont able to compile. what are the jar files i have to use?

    How i have to import this one - import org.sac.crosspather.common.util.AppLogger;

    ReplyDelete
    Replies
    1. ph, Applogger is just used for logging and is not critical.
      You can either use log4j or any logger or just replace all ogger.dolog() with Sytemm.out.println()

      Delete
  14. Thanks Sachin for your solution but it doesn't work for me..
    I get a 401 error code. I have 1.3.17 any ideas ?

    ReplyDelete
  15. Check this post for resolving further 401 Unauthorized issues (after applying Sachin's excellent fix): http://developers.de/blogs/damir_dobric/archive/2009/08/16/configuring-and-troubleshooting-ntlm-and-kerberos-on-windows-7-windows-server-2008-and-iis7.aspx

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. I made some changes and it WORKED, here is my solution:

    1) Copy NTLMEngineImpl class from httpclient4 into your project and resolve problems/dependencies (i removed isProxy() from the code - i dont need it)
    2) Modify JCIFS_NTLMScheme (look at the original solution at the top of the page) source code:
    2.1)
    private String generateType1Msg(String host, String domain) {

    try {
    return new NTLMEngineImpl().generateType1Msg(domain, host);
    } catch (NTLMEngineException e) {
    throw new RuntimeException("generateType1Msg exception");
    }

    }

    2.2)
    private String generateType3Msg(String username, String password, String host,

    String domain, String challenge) {

    try {
    String generateType3Msg = new NTLMEngineImpl().generateType3Msg(username, password, domain, host, challenge);
    return generateType3Msg;
    } catch (NTLMEngineException e) {
    throw new RuntimeException("generateType3Msg exception");
    }
    }

    3) I noticed that domain needs to be provided when creating NTCredential object:
    NTCredentials credentials = new NTCredentials(login, password, "", domain);

    ReplyDelete
    Replies
    1. Congratulations!!! This solution works on Windows Server 2008 R2 (NTLM2). Thanks a lot.

      Delete
  18. hi sachin,
    do you see any reason, why this solution would work perfectly on windows based local jboss server and will fail in unix based jboss server

    HttpClient 4.0.3.jar
    JDK 1.6

    ReplyDelete
  19. It works like a charm!!! Thank you very much!

    ReplyDelete
  20. excellent its works...thanks for the help

    ReplyDelete
  21. This solution did not work for me.
    But I checked out a fork of the same library, with this or similar fix incorporated, built it (it produces EWSJavaAPI_1.2.2) and it worked great: https://github.com/casimirenslip/EWS-Java-API

    ReplyDelete
  22. Hi All,
    Thanks to Sachin that I am able to overcome the support issue with Axis 1.6.2 in my case. However I would like to share a key information that I have found with the JCIFS_NTLMScheme code. This code didnt work as is with an service hosted on IIS 7.5 with only NTLMv2 supported. After lot of troubleshooting and suggestion from my architect had to forcefully use the flag "NtlmFlags.NTLMSSP_NEGOTIATE_NTLM2" in type1 and type3 message constructors instead of using default flags. Hope this will help who has a similar issue.

    ReplyDelete
    Replies
    1. HI Krishna , can you help me with code sample how did u replace flags , i'm facing same situation

      Delete
  23. I am trying to transfer file from Linux to Windows and was using jcifs 1.3.18 jar. It was running dam slow. But when I start using 1.1.11 it is working like a charm. Does any one know reason?

    Thanks in advance.

    ReplyDelete
  24. Hi Sachin,

    Can you please let me know under which license did you publish the above code changes?

    Thanks in Advance.

    ReplyDelete
    Replies
    1. You may consider it as licensed under Apache 2.0. Or let me know if you had another license in mind...

      Delete
  25. Hi Sachin,

    Good work. Can you please let me know is there is any java library like HttpClient or JCIFS which support java 1.4 and NTLM2 ?

    ReplyDelete
  26. Thanks for this post. This resolved my issue.

    ReplyDelete
  27. We are using the same code, we see that it is using the JCIFNTLMScheme, however, we still get a 401 - Credentials not provided in third handshake. We are not able to understand what is going wrong. Any pointers?

    ReplyDelete
  28. Thanks, helped me a lot,
    i need this under old java 1.4.
    For this java you need jcifs in version 1.2.6 (maven dependency version)

    ReplyDelete
  29. Thanks a lot. It really helped me out.
    For your code to compile, I have upgraded my jcifs-1.2.9 to jcifs-1.3.18

    ReplyDelete
  30. This comment has been removed by the author.

    ReplyDelete
  31. Hi Sachin,
    I have created stubs using WSIMPORT and followed the steps that you have mentioned in your post. But it didn't work for me and I am getting Authentication failure. Can you please help. Thanks

    ReplyDelete
  32. mantap bosku ^^ keren dan jangan lupa share terus ya
    cara bermain dominoqq
    situs dominoqq
    agen dominoqq
    http://jilatpantat168.logdown.com/posts/7831649-libraqqtutorial-bermain-dominoqq-pasti-menang
    http://tukangbacot99.logdown.com/posts/7831647-libraqqsitus-agen-dominoqq-terpercaya-dan-berkualitas
    http://masterdomino2019.logdown.com/posts/7831646-ciri-ciri-dan-kelebihan-agen-poker-dominoqq-terpercaya
    https://tukangjilatpantat368.hatenablog.com/entry/2019/05/19/184310
    https://timterhebatdidunia.hateblo.jp/entry/2019/05/19/184542
    https://dominoseo.hatenadiary.jp/entry/2019/05/19/184736

    ReplyDelete
  33. CrownQQ Agen DominoQQ BandarQ dan Domino99 Online Terbesar

    Yuk Buruan ikutan bermain di website CrownQQ

    Sekarang CROWNQQ Memiliki Game terbaru Dan Ternama loh...

    9 permainan :
    => Poker
    => Bandar Poker
    => Domino99
    => BandarQ
    => AduQ
    => Sakong
    => Capsa Susun
    => Bandar 66
    => Perang Baccarat (NEW GAME)

    => Bonus Refferal 20%
    => Bonus Turn Over 0,5%
    => Minimal Depo 20.000
    => Minimal WD 20.000
    => 100% Member Asli
    => Pelayanan DP & WD 24 jam
    => Livechat Kami 24 Jam Online
    => Bisa Dimainkan Di Hp Android
    => Di Layani Dengan 5 Bank Terbaik
    => 1 User ID 9 Permainan Menarik

    Ayo gabung sekarang juga hanya dengan
    mengklick daftar crownqq

    Link Resmi CrownQQ:
    RATUAJAIB.COM
    RATUAJAIB.NET


    DEPOSIT VIA PULSA TELKOMSEL | XL 24 JAM

    BACA JUGA BLOGSPORT KAMI:
    Info CrownQQ
    Cerita Dewasa
    Berita Unik
    Agen BandarQ | Domino99 Online Terbesar
    Berita dan Info Dunia

    Info Lebih lanjut Kunjungi :
    WHATSAPP : +6287771354805
    WHATSAPP2: +855886381279
    LINE : CS_CROWNQQ
    TELEGRAM : +855882357563

    ReplyDelete
  34. NAGAQQ | AGEN BANDARQ | BANDARQ ONLINE | ADUQ ONLINE | DOMINOQQ TERBAIK

    Yang Merupakan Agen Bandarq, Domino 99, Dan Bandar Poker Online Terpercaya di asia hadir untuk anda semua dengan permainan permainan menarik dan bonus menarik untuk anda semua

    Bonus yang diberikan NagaQQ :
    * Bonus rollingan 0.5%,setiap senin di bagikannya
    * Bonus Refferal 10% + 10%,seumur hidup
    * Bonus Jackpot, yang dapat anda dapatkan dengan mudah
    * Minimal Depo 15.000
    * Minimal WD 20.000
    * Deposit via Pulsa TELKOMSEL & XL
    * 6 JENIS BANK ( BCA , BNI, BRI , MANDIRI , CIMB

    Memegang Gelar atau title sebagai AGEN POKER ONLINE Terbaik di masanya

    Games Yang di Hadirkan NagaQQ :
    * Poker Online
    * BandarQ
    * Domino99
    * Bandar Poker
    * Bandar66
    * Sakong
    * Capsa Susun
    * AduQ
    * Perang Bacarrat (New Game)


    Info Lebih lanjut Kunjungi :
    Website : NAGAQQ
    Facebook : Facebook
    WHATSAPP : +855977509035
    Line : Cs_nagaQQ
    TELEGRAM :+855967014811

    BACA JUGA BLOGSPORT KAMI YANG LAIN:
    Cerita seks
    Winner NagaQQ
    Daftar NagaQQ

    ReplyDelete

 
Superblog Directory