REST and SOAP

What is the meaning of RESTful Web Services? 

Explain RESTful web service with examples.

Compare SOAP-based Web Services and RESTful Web Services.

I will talk about

  • basics of REST and REST web services
  • REST vs Soap
    • data type
    • scalability
    • implementation ease
    • specific application need

There are many nice articles talk about the basics of REST. Here I would like to highlight the key points and illustrate with my hands on experience.

REST

REST, representational state transfer is an architectural style, which suggest constraints include Client- Server, stateless, Cacheable, Layered System, Uniform interface and Code on demand. Restful refers to services that conform with the REST constraints.

REST is not a simple concept. The major idea be resources.-oriented.

A Resource can be essentially any coherent and meaningful concept that may be addressed, captured in a representation which is typically a document that captures the current or intended state of a resource. Basic CRUD operations are supported.

Client- Server:
clients and servers are separated by an uniform interface. With separate of concerns of client/server, the client / server is more portable and easier to be replaced.

Stateless:
REST allow Stateless at server side.  servers process requests and return appropriate responses. Requests and responses are built around the transfer of representations of resources.

Client is always be either in transition between application states or “at rest”. State change associates with sending requests.The representation of each state contains links that can be reused for later new state transition. A client in a rest state is able to interact with its user, but creates no load and consumes no per-client storage on the set of servers or on the network.

REST Web Service

A common misunderstanding of REST is that it is limited with HTTP protocol and web services. It is true that REST are mostly implement and described in the context of HTTP.  The author, Roy Fielding is also one of the principal authors of the HTTP specification versions 1.0 and 1.1. However, it is possible to apply REST in other architectural context and other application layer protocols.

REST Web Services is emerging in recent years. It is a collection of resources, with following defined aspects [Wikipedia]

  • the base URI for the web service, such as http://example.com/resources/
  • the Internet media type of the data supported by the web service. This is often JSONXML or YAML but can be any other valid Internet media type.
  • the set of operations supported by the web service using HTTP methods (e.g., GET, PUT, POST, or DELETE).
  • The API must be hypertext driven.

Objective

REST intended to increase scalability by supporting caching which improve response time and save bandwidth and server loading. Not necessary of keep connectivity alive reduce loading further. Also different server can handle the same request

Also, it simplifies component implementation.

According to Fielding,  REST enables intermediate processing by constraining messages to be self-descriptive: interaction is stateless between requests, standard methods and media types are used to indicate semantics and exchange information, and responses explicitly indicate cache ability.

REST also try to re-use existing vocabulary, as in GET/PUT/POST/DELETE for case of HTTP. However there was no way to specify a form method other than GET or POST which make other requests twisted into a POST one. With PUT/DELTE in form supported by HTML 5, the web can be more RESTful. It is also not necessarily to require other discovery mechanism with URI.

Examples

Twitter is a classic exmples of REST Web Service (also for Ruby on Rails).  Its API is based on REST.

For example,

GET statuses/show/:id  will returns a single status, specified by the id parameter below.

while POST statuses/destroy/:id destroys the status specified by the required ID parameter.

Frameworks

Again, the advantage and popularity of REST is brought to us by awesome frameworks.

Ruby on rails is the major one, while other examples include Play!.

They adapt Restful and “conventions over configurations”, to make applications much more clear and greatly reduce the time of development

SOAP vs REST

Scalability

Since the very beginning, SOAP has been considered heavy weight, as in this test. 

First of all, SOAP envelope constitutes the overhead

Also ,it is speculated that the SOAP protocol is introducing a lot of extra overhead in the serialization and deserialization of data which is simply unacceptable for large payload sizes.

SOAP’s performance relative to http is disturbing: since SOAP uses HTTP as its transport one would expect that their performance would converge as the payload size increased

While REST end point is also stateless, state is also being transfer?

http://www.ehow.com/about_6551923_simple-object-access-protocol-work_.html

Data Type

in SOAP, all communication is in XML. web services are not tied to any one operating system or programming language.

However this can also be achieve in other format, such as JSON. In REST there is no such limitation to use XML only. XML is heavy weight and increase the overhead of messages.

Atom Publishing Protocol for publishing to blogs is considered a canonical RESTful protocol.

The possibility to use JSON allow REST to be more light weight. Also end to end integration of technologies like javascript, jsonp to document-oriented database design such as clouchDB is easier. One application to illustrate

Protocol

For Soap, HTTP is used which is proxy-server or firewall friendly and easy to deploy. REST shared the same when HTTP it in use.

As mentioned REST is mostly with HTTP, however it is possible to implement with other application layer protocol, as long as it is possible to transfer meaningful representational state. It can also be extended to Message queue system. Read Restful Queue from ActiveMQ. Soap is a protocol while REST is a style of programming, in the later one there is much more flexibility.

In SOAP, WSDL work as a formal description to define machine to machine interaction. In WSDL 1.1 the http binding is inadequate to describe REST, however WSDL 2.0 can now be used to describe REST as well.

Implementation ease

In traditional web application such as Java EE EJB with SOAP, when there is need to add or modify some parameters, change are often cascading. From UI, ajax and other requests to back-end Java Bean, entries in DB. Although Jsp provide mechanism to render bean, with modern ajax it is not that useful.

The deeper problem with SOAP is strong typing. WSDL use XML Schema and strongly typed messages, which may be  a bad choice for loosely coupled distributed systems.

With REST no special agent is required, standard curl request which can done in browsers. With SOAP usually there is dedicated Library for access. In terms of API upgrade for public web service it is more complex client applications may need new agents to be deployed.

One of the goal of REST is to make web applications more intutitve to develop. Generally true for most cases. Since programmer time is expensive, this is a definitive advantage.

With Restful, there is better forward/backward compatibility since the contract is simpler (involving resource name and allow CRUD) and  less likely to change. The end to end flow  follows convention and adapt to changes. A good read on this will be The Beauty of REST 2004.

REST is simple?

REST based on standard verbs and URI which is intuitive. However, with REST it is sometimes difficult to express resources in a folder-like structure

Also it is hard to model complicate request on resources, e.g. query with different parameters

There are more existing code/ API that  use soap and they are more production-ready. However rest is catching up.

Consider this example  from the JIRA REST API

rest/api/2/search

which search for JIRA issues. one example is /rest/api/2/search?jql&startAt&maxResults&fields&expand

First of all it may not be that intuitive to identify search as a resources. Then there is need to specify a list of parameters, which is similar to non restful WS. Also since there is no side-effect, it supposed to be a get operation, however we can see  that the API itself suggestedIf the JQL query is too large to be encoded as a query param you should instead POST to this resource.”, which is obviously due to limitation of HTTP get request.

SOAP RPC over HTTP, on the other hand, encourages each application designer to define new and arbitrary verbs that suit the usage e.g. getUser. It can be considered more Method-oriented.

Specific Application need

Here suggest that as Soap does retain state and this gives it a special advantage over REST for services that need to preform transactions where multiple calls to a service are need in order to complete a task.

Also  SOAP is more suitable for enterprise level services that implement standard exchange formats in the form of contracts while not supported by REST currently, and SOAP is more reliable with WS-Security, WS-AtomicTransaction and WS-ReliableMessaging.

Both SOAP and REST support SSL but WS-Security more features. Transaction in REST is not fully ACIDic. There is also no built in reliability with SOAP. However, since SOAP message is serialized, it is pass by value instead of reference, which may create synchronization errors and need special care.

Conclusion

In my opinion, REST is becoming more prevailing for most use cases in the world wide web. Meanwhile SOAP should also be in considerations as it may better suit some specific application need.

JSONP

What is JSONP? Explain JSONP. Compare JSONP with JSON.

Background

To understand jsonp, one must first understand what is its intended use.

Same origin policy

An origin is defined by the scheme, host, and port of a URL. In wikipedia entry, there is an example matrix of determining whether the origin is the same. Under this policy, any attempt of requests (by Javascript) on a web page to access a web site must go to the same web site that the web page came from. There is an example in w3’s wiki,

if a document retrieved from http://example.com/doc.html tries to access the DOM of a document retrieved fromhttps://example.com/target.html, the user agent will disallow access because the origin of the first document, (http, example.com, 80), does not match the origin of the second document (https, example.com, 443)

Thereby, this policy prevents a web site’s scripts from accessing methods and properties of scripts used on other sites. This aims to protect the user from attacks where a page contains malicious code to use user’s information. For instance, user may have logged in Facebook, which the code use the credentials to access facebook and work on user’s own for spamming purpose. Cross-site scripting (XSS) is a type of vulnerability to bypass access control like same origin policy.

More details on Same origin policy for different access e.g. Flash, Java can be found here

Problem

With the policy in place, some aspects of non-malicious web development is also being hindered. In short, we cannot make an AJAX (XMLHttpRequest) to a different domain than the curent page.

Workaround

Browsers

Same origin policy can be turned off in some browsers. For example, in Chrome / Firefox. As there is different implementation for browsers, this solution has low compatibility which is crucial in web deelopment.

IFrame

IFrame is to embed an entire HTML document in a page. Sometimes IFrame itself is sufficient for cross-site purpose, for example displaying information from external sites and let the use login in the iframe. Taking Iframe further is to create iframe dynamically and pulling content and set its source to a url in the current domain, same origin is thereby established. This post give a brief illustration to the scenario. However, with iframe it is hard to achieve interaction with the other domain, also lower response time can also be resulted

JSONP to the rescue

JSONP stands for JSON with padding to make use of The HTML <script> element, which is excepted from the same origin policy. It is the pattern of operates on dynamically generated JSON-formatted data from other origins is being retrieved via the script tag.

The JSON content is received when the browser evaluate the script tag element.

For the original data that javascript wanted to dynamically request is in format of JSON, which is light weight and easy to handle. However inject this alone will cause syntax error. Therefore a padding is required.

Padding

By convention, the Padding is typically the name of a callback function

The server response with requested data wrapped with the callback function. The JSON content is received when the browser evaluate the script tag element.

to illustrate,   following is an example (from wikipedia)

<script type="text/javascript"
         src="http://server2.example.com/Users/1234?jsonp=parseResponse">
 </script>

Script element injection

Script Tag Injection

For each new JSONP request, the browser must add a new <script> element, or reuse an existing one. It can be done by DOM manipulation.

In JQuery this functionality is added to the basic json api.  jQuery.getJSON. When URL include the padding, the request is treated as JSONP. It even allow using jsonp to get data and parse it into XML automatically.

is JSONP good?

JSONP is native javascript, which means it is well compatible and light weighted.

However it may prone malicious attack, where when remote server is compromised malicious javascript can be injected to current page.It is being driven that browsers handle request and response with MIME type application/json-p only. Encoding should also be specified

It is popular and used by main web giants, like google in the Google Web Kits. On the otherhand, since JSONP can be use as long as JSON object is response, which is common, here is one example to integrate with many web app like facebook, twitter

My personal experience is that it is often required to get objects response from other domain. Like building a website that integrate with Confluence (wiki tool),  since I dont have the wiki-rendering engine, I either get a lot of mark up language or even I use the standard SOAP/REST Api I get the plain text content, I cannot stimulate what Confluence will show.

I have to fetch it after it has been rendered, which JSONP is a good helper. This experience is the reverse, fetching other site’s information into Confluence.

Comparing JSONP with JSON.

After the discussion, we can see they are not so comparable. JSON itself is a data type, while JSONP is a technique targeting to complement JSON on the cross-domain perspective. Meanwhile, there is no true XMLP since XML cannot be injected to the script tag in the same way.

Proposed Solution over JSONP

After JSON-P, there are some solutions proposed.

Cross-origin resource sharing

CORS is being discussed as to replace JSONP in solving the problem. It is a specification which defines ways for a web server to allow its resources to be accessed by a web page from a different domain (wikipedia)

example from wikipedia.

an additional HTTP headers can be added

Access-Control-Allow-Origin: *

However in order to prevent XDD attack mentioned, a explicit list of permitted origins should be added

Access-Control-Allow-Origin: http://example.com:8080http://foo.example.com

RequireJs and Google Closure Tool

Thanks to my friend’s comment to last post, also thanks to a IE-6-only bug on my site which I spent whole morning in looking for it, I have been deep diving into js again.

Watching the google IO video on the Closure tool, also continue refactoring the javascript in my site with AMD (Asynchronous Module Definition . Quite interested to know more about the concepts behind requirejs, and how is it compared to the Closure tool — whether they are even Comparable. They do have parts in common and integration is perhaps interesting.

I do find some inconvenience bought by requirejs with hands on coding, but perhaps I am using it in a wrong way.

reading this:

http://tomdale.net/2012/01/amd-is-not-the-answer/

and replies on it
http://geddesign.com/post/15994566577/amd-is-the-answer

http://tagneto.blogspot.hk/2012/01/reply-to-tom-on-amd.html (by author of requirejs)

 

also the great official book closure definitive guide on safari books online

Dynamic programming language on web development

  • What are dynamic programming languages, e.g., Ruby, Python?
  • How are they different from traditional programming languages, e.g., Java?
  •  Why are they more productive in web development?

This topic is one of my focus area several months ago when I was building a website. Also this serves as my assignment for the CSIS7306.

I will discuss

  • the definition of dynamic programming languages
  • difference from traditional programming language,
  • why are they more productive in web development
  • comparing some popular languages being used and their development to illustrate

What is Dynamic language

In  Using JavaScript as a Real Programming Language by Tommi Mikkonen and Antero Taivalsaari from Sun Labs 2007, the same problem has been studied with focus on improving javascript.

There suggested the following definition for dynamic language.

Dynamic language is a class of programming languages that share a number of common runtime characteristics that are available in static languages only during compilation, if at all. These behaviors can include the ability to extend the currently running program by loading or generating new code by extending the classes, objects and other definitions of the program, or even by modifying the internals of the language itself, all during program execution. While these behaviors can be emulated in almost any language of sufficient complexity, such behaviors are integral, built-in characteristics of dynamic languages.

The work suggested three main characteristics of dynamic language: Generic, Dynamic typing, Interpretation.

Regarding the characteristics  of dynamic languages comparing to traditional programming languages, I will illustrate by some examples and my experience.

There are numerous languages that are being considered as dynamic. JavaScript, Perl, PHP, Python, Ruby, Groovy, Scala…

Some features that are common but not applicable for all of them.

Features

Metaprogramming

According to wikipedia,

Metaprogramming is writing of programs that write or manipulate other programs (or themselves) as their data or that do part of the work that is otherwise done at run time during compile time.

As in Grooy, it is easy to change the class behavior. Methods can be manipulating at run-time.

For example, this snippets from the official website suggest ways to add a method to the class Book in run-time.

class Book { String title }

Book.metaClass.titleInUpperCase << {-> title.toUpperCase() }

def b = new Book(title:”The Stand”)

Besides, class names, existing methods for String.. can all be configured at run-time. It is also possible to add new classes / scripts.

This actually allow abstract configures from implementation, achieving part of what heavy-weight framework for dependency injection or Aspect-oriented programming frameworks such as Spring do.

It is also powerful when combined with those frameworks.

As in one of my project, I also managed to put groovy script in database and distribute them into target machine and run, with help of “refreshable bean” concept in Spring and activeMq.

This is inspired from the article Groovier Spring, Part 2 

This is available in Ruby, Python, Perl However, not  javascript in its narrow definition. List here

Dynamic typed

A programming language is said to be dynamically typed when the majority of its type checking is performed at run-time as opposed to at compile-time. [wikipedia]

dynamic typing allows duck typing, one is concerned with just those aspects of an object that are used, rather than with the type of the object itself. With this code are much more reusable.

Again not a must, but powerful in many context. There has been a long debate for static typed language vs dynamic typed language. Example like this stackoverflow post

In the context of web development, I will say dynamic typed language greatly speed up development as it evolve with changing implementation. Codes are also cleaner.

Also, for the benefits from static type check, for most dynamic typed language it is also possible to state explicit type checking when necessary.

Closure

Closure is another thing that greatly increase the reusability of code. As a major tool of functional programming, it improves the flexibility of the language and also it keep the global scope tidy. I wont cover much of this here since it is much more on experience and style. JAVA plans to have it only at Java 8, where that may not be a true one.

Dynamic language is better for web Development

Actually nowadays you can hardly find a website without Dynamic language. Most dynamic languages in this context serves as the loose defined scripting language

Client-side programming

This is the most important thing talking about dynamic language in websites.. It is not about to use dynamic language or not, but just where and how much.

Dynamic language is easier to model dynamic website behaviours.

AJAX technologies is mature which handle request and provide dynamic response according to different user behaviours. This asynchrous client side programming is hard to achieve by traditional programming rely on server side. It is possible to use Comet.

The example of JSP, requires different tags and library to do integration for back end and UI, which is overcomplicated and less flexible in my opinion.

Also, closure and help in guarding the scope of variables for the state while providing different features.

It is possible to render different script to client according to different situation. The script client requires can be greatly reduced.

Faster development cycle

This increases developer productivity by reducing scaffolding code when developing web, GUI, database or console applications

As mentioned in previous part, dynamic language is more flexible and cleaner to understand. For web developemnt that need to adapt to rapid changing requirement,  reducing development time will be crucial

This is particular important to handle the front end UI.

No redeployment

Firstly redeployment is not required to add new functionalities.

For traditional ways, like JAVA with tomcat will requires redeployment to compile new code. Although it is just about the startup, it is not suitable for prototyping website as mentioned above, also this will affect the site beahviour and impact the stability. It is required another server for flip-flop to keep the site running.

Same data structure from UI to back end

Although this can also be achieve with JAVA EE and JSP, that is not as seamless. As mentioned most UI used dynamic language like javascript for dynamic behaviour, lots of paring is required to do the end to end integration. For example, front end send back a JSON object, which will require to parse by the JAVA servlet to Java Objects with mapping. There exists lots of libraries like GSON to help, but still manual work has to be done on customizing different types of complex objects as it is not dynamic typed. Error due to Reflection lack of public constructors are not uncommon.

There is another step to convert those objects back the the back-end database. Hibernate is there but still that is one another configuration.

I will argue that with dynamic language in both front end and back end, it is possible to use the same data structure. There are Document-oriented database such as CouchDB, which use JSON for storage. This provides mechanism for end to end using JSON, simple but powerful. From experience lots of error can be reduced in parsing code.

Performance?

Regarding the production performance, many argue that dynamic language is less productive, as there are extra checking to be done in run-time. However for the discussion it is important to note  Hiphop for PHP and Google V8 Engine, examples of write in dynamic language, where traditional language allow them run faster.

Current Developement

I will talk about some languages, framework and their development.

1) Javascript and its development

Javascript can definitely speaks for the place of dynamic language in web development. Contradict to what most people thought, Javascript is not a toy language that only capable for showing consecutive alerts of “Hello”, and some small icons flying around your cursor, as what is popular 15 years ago. In recent years there has been a great development over javascript.

Again in the work by Tommi et al here is the recap of javascript:

JavaScript is a typical dynamic language in the sense that variables in JavaScript do not need to be introduced before their use and that the types of variables are resolved dynamically during execution. JavaScript allows function definitions and other code to be modified while the program is running. The execution model of JavaScript is based on interpretation of source code. In contrast with less dynamic languages such as the Java programming language, there is no public intermediate representation format such as class files or binary files. In summary, JavaScript manifests all the three key characteristics of dynamic languages listed in Section 2.1.

Java script is dynamic-typed and it is object -oriented,  to be exact, following a prototype pattern. A long but awesome discussion on this can be find here

I would say it is some concepts is more complicated than traditional language but using it is handy and flexible. 

It is true that there are many design flaws in the language itself. Some weired behaviour documented in the Javascript garden. However, Javascript still stay as a simple,powerful language with the its good parts. This single book is the bible of writing awesome javascript.

Recent development extend Javascript in server-side programming (Node.js) or other areas outside browser (CommonJS), better standard to modularize the code to become reusable like Asynchronous Module Definition, with example as (RequireJS).  Lots of libraries help to achieve more with less code and address browser compatibility issues like JQuery.

Yes, I agree that there should be a domain name .js for these sites.

Unlike mysterious error you will see in IE6, debugging and development is now much easier with Web Developer tools in Firefox or Chrome, which even support hot code replace.

2) GRails and Play! framework

Both inspird by Ruby on rails, the two frameworks are very good examples about integrating dynamic language with traditional language in web development.

With increasing complexity in web development which is now also more mature, I will suggest framework is of crucial  importance. For past web MVC frameworks, it has been shown that J2EE EJB has its design flaws, while spring web MVC might be too heavy-weighted.

These frameworks adapt a “convention over configuration” approach, which help developers to build a easy-to-maintain application in short time.

It has to be admitted that not only dynamic language, but many other components and ideas such as RESTful, Behaviour driven development, LESS CSS, together with community support including Heroku and github, consituted such great frameworks. However the language itself is crucial for linking everything together and make it usable for developers

Developers of those framework do a great job that Java developers can use their familar language while providing dynamic behaviour support.

Grails use groovy, which can be translate to bytecode running in JVM. This allows extending legacy Java system with dynamic behaviour in an easy manner.

Play! framework support automatic source loading and use Comet Socket and WebSockets to enhance dynamic behaviour compared to traditional Java. It also use Scala as the template engine, which integrate easily with Java objects in the view.

Conclusion

This has been a long post, but for sure there are even more discussion that can be going on as this is a really big topic. The simple conclusion is that it is not about using dynamic language or not, but how much it is going to replace in the traditional way of web development.