Monday, December 8, 2014

What is POODLE Vulnerability and how does it affect you ?

Originally Published at SaurzCode :- [http://saurzcode.in/2014/12/poodle-vulnerability-affects/]


What is POODLE?


It stands for “Padding Oracle On Downgraded Legacy Encryption.”.This means a protocol downgrade that allows exploits on an outdated form of encryption.It was first explained in Google Security Advisory.

Poodle
Poodle
To explain this in simpler terms, if an attacker using a Man-In-The-Middle attack can take control of a router at a public hotspot, they can force your browser to downgrade to SSL 3.0 (an older protocol) instead of using the much more modern TLS (Transport Layer Security), and then exploit a security hole in SSL to hijack your browser sessions and get useful information out of your session cookies to your accounts in Google,Yahoo or your bank too. Since this problem is in the protocol, anything that uses SSL is affected. As long as both the server and the client (web browser) support SSL 3.0, the attacker can force a downgrade in the protocol, so even if your browser tries to use TLS, it ends up being forced to use SSL instead. The only answer is for either side or both sides to remove support for SSL, removing the possibility of being downgraded. Impact


The POODLE attack can be used against any system or application that supports SSL 3.0 with CBC mode ciphers. This affects most current browsers and websites, but also includes any software that either references a vulnerable SSL/TLS library (e.g. OpenSSL) or implements the SSL/TLS protocol suite itself. By exploiting this vulnerability in a likely web-based scenario, an attacker can gain access to sensitive data passed within the encrypted web session, such as passwords, cookies and other authentication tokens that can then be used to gain more complete access to a website (impersonating that user, accessing database content, etc.)



How Can We Solve the Problem?


As a user, you want to protect yourself from attacks, and the best way to do that is to disable SSL 3 in your browser. As a web site operator, you should disable SSL 3 on your servers as soon as possible. You need to do this even if you support the most recent TLS version because an active MITM attacker can force browsers to downgrade their connections all the way down to SSL 3, which can then be exploited

Microsoft Advisory -


https://technet.microsoft.com/en-us/library/security/3009008.aspx

References


Sunday, November 2, 2014

More Effective Java With Joshua Bloch

Originally Shared at : [http://saurzcode.in/2014/11/02/more-effective-java-with-joshua-bloch/]

Many of us already agree how great the book Effective Java by Joshua Bloch is and it's a must read for every Java Developer out there whether you have just started or working for a while.While reading the book and researching on some of the Items listed in the book, I came across this Interview with Joshua Bloch Link at Oracle , in which he speaks about some of the great things in the book and shares his knowledge on some great topics in the language.This should be a good read for someone interested to explore more while reading this book or afterwards -

Here is the link -

http://www.oracle.com/technetwork/articles/java/bloch-effective-08-qa-140880.html




Also take a looks at -

Monday, September 29, 2014

Java : What does finalize do and How?

Originally shared at : [http://saurzcode.in/2014/09/29/java-what-does-finalize-do-and-how/]

Finalize method in Object class is often a point of discussion whether to be used or not ? Below are some of the pointers on Finalize method

  • When It is Called : Called by the garbage collector on an object when garbage collection determines that there are no more references to the object. A subclass overrides the finalize method to dispose of system resources or to perform other cleanup.

  • The general contract of finalize is that it is invoked if and when the JavaTM virtual machine has determined that there is no longer any means by which this object can be accessed by any thread that has not yet died, except as a result of an action taken by the finalization of some other object or class which is ready to be finalized.

  • The finalize method may take any action, including making this object available again to other threads; the usual purpose of finalize, however, is to perform cleanup actions before the object is irrevocably discarded. For example, the finalize method for an object that represents an input/output connection might perform explicit I/O transactions to break the connection before the object is permanently discarded.

  • The finalize method of class Object performs no special action; it simply returns normally. Subclasses of Object may override this definition.


protected void finalize() throws Throwable { }


  • The Java programming language does not guarantee which thread will invoke the finalize method for any given object. It is guaranteed, however, that the thread that invokes finalize will not be holding any user-visible synchronization locks when finalize is invoked. If an uncaught exception is thrown by the finalize method, the exception is ignored and finalization of that object terminates.

  • After the finalize method has been invoked for an object, no further action is taken until the Java virtual machine has again determined that there is no longer any means by which this object can be accessed by any thread that has not yet died, including possible actions by other objects or classes which are ready to be finalized, at which point the object may be discarded.

  • The finalize method is never invoked more than once by a Java virtual machine for any given object.

  • Any exception thrown by the finalize method causes the finalization of this object to be halted, but is otherwise ignored.

  • In general it's best not to rely on finalize() to do any cleaning up etc., because a object may not be eligible for GC during the lifetime of the application and resources might not get closed and can cause the resource exhaustion.

  • If overriding finalize() it is good programming practice to use a try-catch-finally statement and to always call super.finalize(). This is a safety measure to ensure you do not inadvertently miss closing a resource used by the objects calling class
    protected void finalize() throws Throwable {
    try {
    close
    (); // close open files
    } finally {
    super.finalize();
    }
    }



References : http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html

Thursday, September 11, 2014

What can I learn right now in just 10 minutes that could be useful for the rest of my life?

Answer by Vishnu Haridas:
This one I discovered recently: If you get an unusable headphones, don't throw it away. You can cut & remove the wire, and use the TRS jack as the FM antenna for your smartphone.

All you need is to plug-in this TRS jack into your phone's headphones plug, and open the FM radio app, then start listening through your loudspeaker.

How it works: The headphones wire works as the FM antenna for mobile phones. Usually FM transmission will have a very strong signal, which needs a small piece of wire to receive the signal.

What can I learn right now in just 10 minutes that could be useful for the rest of my life?

Tuesday, August 19, 2014

How to configure Swagger to generate Restful API Doc for your Spring Boot Web Application ?

Originally Posted here - http://saurzcode.in/2014/08/how-to-configure-swagger-to-generate-restful-api-doc-for-your-spring-boot-web-application/
“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.”
- Martin Fowler

What is Swagger ?


Swagger is a specification and complete framework implementation for describing, producing, consuming, and visualizing RESTful web services. The goal of Swagger is to enable client and documentation systems to update at the same pace as the server. The documentation of methods, parameters, and models are tightly integrated into the server code, allowing APIs to always stay in sync.

Why is Swagger useful?


The Swagger framework simultaneously solves server, client, and documentation/sandbox needs.

With Swagger's declarative resource specification, clients can understand and consume services without knowledge of server implementation or access to the server code. The Swagger UI framework allows both developers and non-developers to interact with the API in a sandbox UI that gives clear insight into how the API responds to parameters and options.

Swagger happily speaks both JSON and XML, with additional formats in the works.

How to Enable Swagger in your Spring Boot Web Application ?


Step 1 : Include Swagger Spring MVC dependency in Maven


<dependency>
<groupId>com.mangofactory</groupId>
<artifactId>swagger-springmvc</artifactId>
<version>0.8.8</version>
</dependency>

Step 2 : Create Swagger Java Configuration



  • Use the @EnableSwagger annotation.

  • Autowire SpringSwaggerConfig.

  • Define one or more SwaggerSpringMvcPlugin instances using springs @Bean annotation.


[gist https://gist.github.com/saurzcode/9dcee7110707ff996784/]

Step 3 : Create Swagger UI using WebJar


<repository>
<id>oss-jfrog-artifactory</id>
<name>oss-jfrog-artifactory-releases</name>
<url>http://oss.jfrog.org/artifactory/oss-release-local</url>
</repository>

<dependency>
<groupId>org.ajar</groupId>
<artifactId>swagger-spring-mvc-ui</artifactId>
<version>0.1</version>
<scope>compile</scope>
</dependency>

Verify the API Configuration at  - http://localhost:8080/api-docs

You can see the Swagger API Docs at  http://localhost:8080/index.html

[caption id="attachment_674" align="aligncenter" width="640"]Swagger API Doc Swagger API Doc[/caption]

Complete project is available at GitHub.

https://github.com/saurzcode/saurzcode-swagger-spring/

References :





You may also like :-

Saturday, August 9, 2014

How to Setup Realtime Alalytics over Logs with ELK Stack : ElasticSearch, Logstash, Kibana?

Once we know something, we find it hard to imagine what it was like not to know it.

- Chip & Dan Heath, Authors of Made to Stick, Switch

Originally Shared at : [http://saurzcode.in/2014/08/09/how-to-setup-realtime-alalytics-over-logs-with-elk-stack/]

What is the ELK stack ?


The ELK stack is ElasticSearch, Logstash and Kibana. These three provide a fully working real-time data analytics tool for getting wonderful information sitting on your data.

ElasticSearch

ElasticSearch,built on top of Apache Lucene, is a search engine with focus on real-time analysis of the data, and is based on the RESTful architecture. It provides standard full text search functionality and powerful search based on query. ElasticSearch is document-oriented/based and you can store everything you want as JSON. This makes it powerful, simple and flexible.

Logstash

Logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use.In ELK Stack logstash plays an important role in shipping the log and indexing them later which can be supplied to Elastic Search.

Kibana

Kibana is a user friendly way to view, search and visualize your log data, which will present the data stored from Logstash into ElasticSearch, in a very customizable interface with histogram and other panels which provides real-time analysis and search of data you have parsed into ElasticSearch.

How do I get it  ?

http://www.elasticsearch.org/overview/elkdownloads/

How do they work together ?

Logstash is essentially a pipelining tool. In a basic, centralized installation a logstash agent, known as the shipper, will read input from one to many input sources and output that text wrapped in a JSON message to a broker. Typically Redis, the broker, caches the messages until another logstash agent, known as the collector, picks them up, and sends them to another output. In the common example this output is Elasticsearch, where the messages will be indexed and stored for searching. The Elasticsearch store is accessed via the Kibana web application which allows you to visualize and search through the logs. The entire system is scalable. Many different shippers may be running on many different hosts, watching log files and shipping the messages off to a cluster of brokers. Then many collectors can be reading those messages and writing them to an Elasticsearch cluster.

[caption id="attachment_644" align="aligncenter" width="640"]ELK (E)lasticSearch (L)ogstash  (K)ibana (The ELK Stack)[/caption]

How do i fetch useful information out of logs ? 

Fetching useful information from logs is one of the most important part of this stack and is being done in logstash using its grok filters and a set of input , filter and output plugins which helps to scale this functionality for taking various kinds of inputs ( file,tcp, udp, gemfire, stdin, unix, web sockets and even IRC and twitter and many more) , filter them using (groks,grep,date filters etc.) and finally write ouput to ElasticSearch,redis,email,HTTP,MongoDB,Gemfire , Jira , Google Cloud Storage etc.

A bit more about Log Stash

grok

Filters 

Transforming the logs as they go through the pipeline is possible as well using filters. Either on the shipper or collector, whichever suits your needs better. As an example, an Apache HTTP log entry can have each element (request, response code, response size, etc) parsed out into individual fields so they can be searched on more seamlessly. Information can be dropped if it isn’t important. Sensitive data can be masked. Messages can be tagged. The list goes on.

e.g.

[gist https://gist.github.com/saurzcode/da3b31f0496b5feba8c9 /]


Above example takes input from an apache log file applies a grok filter with %{COMBINEDAPACHELOG}, which will index apache logs information on fields and finally output to Standard Output Console.

Writing Grok Filters

Writing grok filters and fetching information is the only task that requires some serious efforts and if done properly will give you great insights in to your data like Number of Transations performed over time, Which type of products have most hits etc.

Below links will help you a lot in writing grok filters and test them with ease -

Grok Debugger

http://grokdebug.herokuapp.com/

Grok Patterns Lookup

https://github.com/elasticsearch/logstash/tree/v1.4.2/patterns

References -

  • http://www.elasticsearch.org/overview/

  • http://logstash.net/

  • http://rashidkpc.github.io/Kibana/about.html






Related Articles :-

Hadoop Certification

Getting Started with Apache Pig

Hadoop Reading List

Sunday, June 29, 2014

Hadoop : Getting Started with Pig

What is Pig?


Pig is a high level scripting language that is used with Apache Hadoop. Pig enables data analysts to write complex data transformations without knowing Java. Pig’s simple SQL-like scripting language is called Pig Latin, and appeals to developers already familiar with scripting languages and SQL.Pig Scripts are converted into MapReduce Jobs which runs on data stored in HDFS (refer to the diagram below).


Through the User Defined Functions(UDF) facility in Pig, Pig can invoke code in many languages like JRuby, Jython and Java. You can also embed Pig scripts in other languages. The result is that you can use Pig as a component to build larger and more complex applications that tackle real business problems.



Pig Architecture


Pig Achitecture



How Pig is being Used ?



  • Rapid prototyping of algorithms for processing large data sets.

  • Data Processing for web search platforms.

  • Ad Hoc queries across large data sets.

  • Web log processing.


Pig Elements


Pig consists of three elements -




  • Pig Latin

    • High level scripting language

    • No Schema

    • Translated to MapReduce Jobs



  • Pig Grunt Shell

    • Interactive shell for executing pig commands.



  • PiggyBank

    • Shared repository for User defined functions (explained later).




Pig Latin Statements 


Pig Latin statements are the basic constructs you use to process data using Pig. A Pig Latin statement is an operator that takes a relation as input and produces another relation as output(except LOAD and STORE statements).


Pig Latin statements are generally organized as follows:




  • A LOAD statement to read data from the file system.

  • A series of "transformation" statements to process the data.

  • A DUMP statement to view results or a STORE statement to save the results.


Note that a DUMP or STORE statement is required to generate output.




  • In this example Pig will validate, but not execute, the LOAD and FOREACH statements.
    A = LOAD 'student' USING PigStorage() AS (name:chararray, age:int, gpa:float);
    B = FOREACH A GENERATE name;


  • In this example, Pig will validate and then execute the LOAD, FOREACH, and DUMP statements.
    A = LOAD 'student' USING PigStorage() AS (name:chararray, age:int, gpa:float);
    B = FOREACH A GENERATE name;
    DUMP B;
    (John)
    (Mary)
    (Bill)
    (Joe)



Storing Intermediate Results


Pig stores the intermediate data generated between MapReduce jobs in a temporary location on HDFS. This location must already exist on HDFS prior to use. This location can be configured using the pig.temp.dir property.



Storing Final Results


Use the STORE operator and the load/store functions to write results to the file system ( PigStorage is the default store function).


Note: During the testing/debugging phase of your implementation, you can use DUMP to display results to your terminal screen. However, in a production environment you always want to use the STORE operator to save your results.



Debugging Pig Latin


Pig Latin provides operators that can help you debug your Pig Latin statements:




  • Use the DUMP operator to display results to your terminal screen.

  • Use the DESCRIBE operator to review the schema of a relation.

  • Use the EXPLAIN operator to view the logical, physical, or map reduce execution plans to compute a relation.

  • Use the ILLUSTRATE operator to view the step-by-step execution of a series of statements.


What is Pig User Defined Functions (UDFs) ?


Pig provides extensive support for user-defined functions (UDFs) as a way to specify custom processing. Functions can be a part of almost every operator in Pig.UDF is very powerful functionality to do many complex operations on data.The Piggy Bank is a place for Pig users to share their functions(UDFs).


Example:



REGISTER saurzcodeUDF.jar;
A = LOAD 'employee_data' AS (name: chararray, age: int, designation: chararray);
B = FOREACH A GENERATE saurzcodeUDF.UPPER(name);
DUMP B;

This article was just a Getting Started Article on Pig , I will cover further details about How to to Write Pig Latin commands for some basic operations like JOIN,FILTER,GROUP, ORDER etc. , also how to make your own UDFs for processing on Hadoop cluster.


References :-




  • http://pig.apache.org






 Related articles :


Monday, May 19, 2014

String Interning - What ,Why and When ?

What is String Interning 

String Interning is a method of storing only one copy of each distinct String Value, which must be immutable.

In Java String class has a public method intern() that returns a canonical representation for the string object. Java's String class privately maintains a pool of strings, where String literals are automatically interned.




When the intern() method is invoked on a String object it looks the string contained by this String object in the pool, if the string is found there then the string from the pool is returned. Otherwise, this String object is added to the pool and a reference to this String object is returned.



The intern() method helps in comparing two String objects with == operator by looking into the pre-existing pool of string literals, no doubt it is faster than equals() method. The pool of strings in Java is maintained for saving space and for faster comparisons. Normally Java programmers are advised to use equals(), not ==, to compare two strings. This is because == operator compares memory locations, while equals() method compares the content stored in two objects.


Why and When to Intern ?


Thought Java automatically interns all Remember that we only need to intern strings when they are not constants, and we want to be able to quickly compare them to other interned strings. The intern() method should be used on strings constructed with new String() in order to compare them by == operator.


Let's take a look at the following Java program to understand the intern() behavior.




[code language="java"]
public class TestString {

public static void main(String[] args) {
String s1 = "Test";
String s2 = "Test";
String s3 = new String("Test");
final String s4 = s3.intern();
System.out.println(s1 == s2);
System.out.println(s2 == s3);
System.out.println(s3 == s4);
System.out.println(s1 == s3);
System.out.println(s1 == s4);
System.out.println(s1.equals(s2));
System.out.println(s2.equals(s3));
System.out.println(s3.equals(s4));
System.out.println(s1.equals(s4));
System.out.println(s1.equals(s3));
}

}


//Output
true
false
false
false
true
true
true
true
true
true


[/code]

Friday, May 9, 2014

SOAP Webservices Using Apache CXF : Adding Custom Object as Header in Outgoing Requests

What is CXF?


Apache CXF is an open source services framework. CXF helps you build and develop services using frontend programming APIs, like JAX-WS and JAX-RS. These services can speak a variety of protocols such as SOAP, XML/HTTP, RESTful HTTP, or CORBA and work over a variety of transports such as HTTP, JMS etc.


How CXF Works?


As you can see here and here, how CXF service calls are processed,most of the functionality in the Apache CXF runtime is implemented by interceptors. Every endpoint created by the Apache CXF runtime has potential interceptor chains for processing messages. The interceptors in the these chains are responsible for transforming messages between the raw data transported across the wire and the Java objects handled by the endpoint’s implementation code.


Interceptors in CXF


When a CXF client invokes a CXF server, there is an outgoing interceptor chain for the client and an incoming chain for the server. When the server sends the response back to the client, there is an outgoing chain for the server and an incoming one for the client. Additionally, in the case of SOAPFaults, a CXF web service will create a separate outbound error handling chain and the client will create an inbound error handling chain.


The interceptors are organized into phases to ensure that processing happens on the proper order.Various phases involved during the Interceptor chains are listed in CXF documentation here.


Adding your custom Interceptor involves extending one of the Abstract Intereceptor classes that CXF provides, and providing a phase when that interceptor should be invoked.


AbstractPhaseInterceptor class - This abstract class provides implementations for the phase management methods of the PhaseInterceptor interface. The AbstractPhaseInterceptor class also provides a default implementation of the handleFault() method.


Developers need to provide an implementation of the handleMessage() method. They can also provide a different implementation for the handleFault() method. The developer-provided implementations can manipulate the message data using the methods provided by the generic org.apache.cxf.message.Message interface.


For applications that work with SOAP messages, Apache CXF provides an AbstractSoapInterceptor class. Extending this class provides the handleMessage() method and the handleFault() method with access to the message data as an org.apache.cxf.binding.soap.SoapMessage object. SoapMessage objects have methods for retrieving the SOAP headers, the SOAP envelope, and other SOAP metadata from the message.


Below piece of code will show, how we can add a Custom Object as Header to an outgoing request –


Spring Configuration




[code language="xml"]
<jaxws:client id="mywebServiceClient"
serviceClass="com.saurzcode.TestService"
address="http://saurzcode.com:8088/mockTestService">

<jaxws:binding>
<soap:soapBinding version="1.2" mtomEnabled="true" />
</jaxws:binding>
</jaxws:client>
<cxf:bus>
<cxf:outInterceptors>
<bean class="com.saurzcode.ws.caller.SoapHeaderInterceptor" />
</cxf:outInterceptors>
</cxf:bus>
[/code]


Interceptor :-

[code language="java"]
public class SoapHeaderInterceptor extends AbstractSoapInterceptor {

public SoapHeaderInterceptor() {

super(Phase.POST_LOGICAL);

}

@Override
public void handleMessage(SoapMessage message) throws Fault {

List<Header> headers = message.getHeaders();

TestHeader testHeader = new TestHeader();

JAXBElement<TestHeader> testHeaders = new ObjectFactory()

.createTestHeader(testHeader);

try {

Header header = new Header(testHeaders.getName(), testHeader,

new JAXBDataBinding(TestHeader.class));

headers.add(header);

message.put(Header.HEADER_LIST, headers);

} catch (JAXBException e) {

e.printStackTrace();

}

}


[/code]

Monday, April 21, 2014

Free Online Hadoop Trainings

Hadoop and Big Data are becoming the new hot trends of the industry , being the most sought out skills in the market.There are various vendors and online training providers coming up with a nice explanation of some of the core concepts underlying Hadoop frameworks like Mapreduce,HDFS and various components involved in Hadoop Ecosystem.I will try to list down some of these resources here -

1. Udacity - Cloudera came with this nice course named "Intro to Hadoop and MapReduce" .This provides a nice explanation of the core concepts and internal working of hadoop components embedded with quizzes around each concept and some good handson exercies.They also provide VM for training purpose, which can be used to run example questions and to solve quizzes and exams for the courses.

Goals -

  • How Hadoop fits into the world (recognize the problems it solves)
  • Understand the concepts of HDFS and MapReduce (find out how it solves the problems)
  • Write MapReduce programs (see how we solve the problems)
  • Practice solving problems on your own

Prerequisites -

Some basic programming knowledge and a good interest in learning :)

2. Introduction to Mapreduce Programming

3. Moving Data in to Hadoop

Tuesday, February 4, 2014

Recommended Readings for Hadoop

I am writing this series to mention some of the recommended reading to understand Hadoop , its architecture, minute details of cluster setup etc.

Understanding Hadoop Cluster Setup and Network - Brad Hedlund, with his expertise in Networks, provide minute details of cluster setup, data exchange mechanisms of a typical Hadoop Cluster Setup.

MongoDB and Hadoop - Webinar by Mike O'Brien,Software Engineer, MongoDB on how MongoDB and Hadoop can be used together , using core MapReduce and Pig and Hive as well.

[slideshare id=25070101&doc=hadoop-webinar-130808141030-phpapp01]

Please post comments if you have come across some great article/webinar link, which explains things in great details with ease.