Well, at the end of the movie I had a little discussion, and my take was that it really was not all that confusing, if you thought of it in terms of recursion. Add a couple of weeks and a little inspiration from another discussion thread, plus several years of reading XKCD; and voila I drew a comic. Enjoy.

req.getRequestDispatcher("/accessDenied.html").forward(request, response);

However, a Java web container is multi-threaded, and this merely spawns a new thread, and the server processes both the JSP and the page that has been forwarded to simultaneously. If one looks at the logging output from the server, or intercepts the HTTP traffic, using, for example, a packet sniffer (which are commonplace enough to exist as Firefox extensions), one can see this happening live. This method therefore, clearly raises some nasty security concerns.

Furthermore, from a software engineering or architectural design point of view, using a JSP as a filter violates MVC design principles, as doing so would make the JSP act as a controller as well as a view at the same time.

The proper way to go about implementing restricted access to certain pages is to use filters from the Servlet API, and then configuring the web application to map select pages to these filters. This method utilises additional capability provided by web container that runs on the server. The web container constructs filter chains based on the filter mappings defined. Most crucially, these filters allow the authentication logic and consequent branching to be executed prior to the JSP being called;

Create an abstract class `LoginFilter`.

    public abstract class LoginFilter implements javax.servlet.Filter {
        protected ServletContext servletContext
    
        public void init(FilterConfig filterConfig) {
            servletContext = filterConfig.getServletContext();
        }
    
        public void doFilter(
            ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {
            HttpServletRequest req = (HttpServletRequest)request;
            HttpServletResponse resp = (HttpServletResponse)response;
        
            if (!isAuth()) {
                resp.sendError(HttpServletResponse.SC_UNAUTHORIZED);
                return; //break filter chain, requested JSP/servlet will not be executed
            }
        
            //propagate to next element in the filter chain, ultimately JSP/ servlet gets executed
            chain.doFilter(request, response);        
        }
    
        /**
         * logic to accept or reject access to the page, check log in status
         * @return true when authentication is deemed valid
         */
        protected abstract boolean isAuth();
    
    }

Subclass the `LoginFilter` as a concrete class, `MemberLoginFilter`, by implementing the `isAuth` method. In this case add logic that determines whether a user is currently logged in. This post does not cover how this is done; in a nutshell, this would typically involve retrieving objects from the current HTTP session, and determining whether a user has logged in during the current session by comparing the values or states of these objects.

Now edit the `web.xml` configuration for the web application archive (WAR), to make the web app aware of the filter and what pages it should map to.

<filter>
    <description>Requires user to log in as a member</description>
    <filter-name>MemberLoginFilter</filter-name>
    <filter-class>some.package.MemberLoginFilter</filter-class>
</filter>
<filter-mapping>
    <filter-name>MemberLoginFilter</filter-name>
    <url-pattern>/someRestrictedPage.jsp</url-pattern>
</filter-mapping>

Add as many filter mappings as necessary for each page that needs to have access to it restricted to logged-in members only; use file name globs to match multiple files.

Also, still editing `web.xml` configuration, tell the web app how to handle access denied error messages in a more user friendly way, by adding the following node to the settings:

<error-page>
    <error-code>401</error-code>
    <location>/accessDenied.html</location>
</error-page>

The HTTP 401 error code referenced here corresponds to `HttpServletResponse.SC_UNAUTHORIZED` in `LoginFilter.doFilter(..)`.
This completes the loop: when the filter chain is broken because `LoginFilter.isAuth()` returns false, instead of getting a garish HTTP 401, the user will see an access denied page instead, which you can edit to inlcude a user friendly message.

If `LoginFilter.isAuth()` returns true, the filter chain continues execution, and assuming that there are no more filters in the filter chain, or all of the downstream filters in the filter chain pass, the requested JSP or servlet gets displayed.

Subversion diff ignoring whitespace

svn diff --diff-cmd /usr/bin/diff -x -bBw -r9999 > r9999.diff

Compares the local copy with a particular revision; and uses the local diff command instead of the one built into the Subversion client.

The `-x` flags that the following option (in this case `-bBw`) is for the local diff command, instead of the Subversion command.

Find files containing a particular string recursively in a directory

find . -name \*.java -exec grep -n "searchstringregex" {} /dev/null \;

Lists all files within the current directory (recursively) which match a file glob pattern, and contain at least one line that matches a regular expression. Special characters in the file glob need to be escaped (e.g. `\*.java`), and the regex should be quoted.

The purpose of including `/dev/null` is to give the `grep` command more than one input file (per iteration), to force grep to output not just the matching string, but also the file name. This would be redundant if we are lookign at just one file, however it is quite useful, and necessary, here since we are grep’ping many files at once.

find . -name \*.java -exec grep -c "searchstringregex" {} /dev/null \; | grep -v :0$

A very slight variation on the previous command, this time with the number of occurrences of the regex in each file (`-c`) instead of the contents of each matching line. The output is filtered by piping it through another `grep` to remove items with a count of zero.

Guide to understanding the basics of indexing and the various merges in line with their effects on execution plans

How-to manual from Oracle themselves on how to use autotrace in SQLPlus

The most complete reference on Oracle hints I’ve come across

A collection of specific, and advanced, query tuning techniques

Also, here is the 1979 seminal paper from P. Selinger of IBM, which forms the roots of access plan optimisation in most current day databases; and addresses the basic concepts that need to be taken into account when optimising queries.
Access path selection in a relational database management system

This one here does not have anything to do directly with query optimisation, but I found that attempting to optimise queries without first understanding the algorithms behind them is a fool’s errand. So here is a great post on the basic algorithms behind nested loop join, merge join and hash join (code in C#).

Fortunately, it is much easier than Windows, everything is achieved with a single shell script.

#!/bin/bash
# Installs python-iview including all dependencies
# Script assumes /usr/bin in on the path (is with default installs)
# Brendan Graetz 20100521
#
# python-iview by Jeremy Visser: http://jeremy.visser.name/2009/08/30/python-iview/

#install bazaar and python library
sudo apt-get install bzr
sudo apt-get install python-beautifulsoup

#install dev dependencies in order to be able to compile
sudo apt-get install libssl-dev
sudo apt-get install zlib1g-dev

#download rtmpdump source
wget http://rtmpdump.mplayerhq.hu/download/rtmpdump-2.2d.tgz ~/Downloads/rtmpdump-2.2d.tgz

#compile rtmpdump & create softlink on path
cd /usr/local/
tar -xzf ~/Downloads/rtmpdump-2.2d.tgz
cd rtmpdump-2.2d
make posix
sudo ln -s `pwd`/rtmpdump /usr/bin/rtmpdump

#check out python iview from repository
cd /usr/local/
bzr branch http://jeremy.visser.name/bzr/python-iview/ ./python-iview

#create softlinks on path
sudo ln -s `pwd`/iview-gtk /usr/bin/iview-gtk
sudo ln -s `pwd`/iview-cli /usr/bin/iview-cli
sudo ln -s /usr/bin/iview-gtk /usr/bin/iview

The only somewhat tricky bit here is that you need to compile rtmpdump, because it does not distribute binaries (not .deb‘s anyway). For that you will need to install a couple of dependencies which are not immediately obvious: libssl-dev and zlib1g-dev.

sed -e 's/Adobe/Apple/g' -e 's/Flash/closed/g' -e 's/third party/proprietary/g' apple.txt
cat apple.txt

(substitute file names accordingly)

There is no simpler way to expose hypocrisy than simple text substitution. Enjoy.

(slightly modified version of the suggestion in this post)

Only 10% of programmers can write a binary search

Shocking indeed

Better yet, he has a coding challenge for you to take part in (post answers as comments):

Here are the rules:

1. Use whatever programming language you like.
2. No cutting, pasting or otherwise copying code.  Don’t even look at other binary search code until you’re done.
3. I need hardly say, no calling bsearch(), or otherwise cheating
4. Take as long as you like — you might finish, and feel confident in your code, after five minutes; or you’re welcome to take eight hours if you want (if you have the time to spare).
5. You’re allowed to use your compiler to shake out mechanical bugs such as syntax errors or failure to initialise variables, but …
6. NO TESTING until after you’ve decided your program is correct.
7. Finally, the most important one: if you decide to begin this exercise, then you must report — either to say that you succeeded, failed or abandoned the attempt.  Otherwise the figures will be skewed towards success.

If you haven’t yet, try it. Read just the article, but do not look at the others’ comments just yet.

After you have attempted this, there is his redux, on why coding exercises like this matter; and his redux of the redux, on why you should not test while attempting exercises like this. They are all worth a good read.

So, on a whim, I decide plug it in. I would encourage you to try it too:

import antigravity

Spoiler: python antigravity easter egg as evidenced in python’s source code (circa 2008)

Java Web Start

The Java Network Launch Protocol (JNLP) specifies how to deliver a desktop application over the internet. A webpage may link to a *.jnlp file, and when you download it, the JVM is invoked and will parse the file and download or stream the Java classes of the application, and run the application as if it were on your desktop (not within the browser).

Sun has a full-blown tutorial on how to develop, deploy and run Java Web Start applications – it is a significantly trickier task to accomplish than with standard JAR file distributions; and rightly so, because applications deployed over networks need tighter security at several different levels.

By default, Java applications deployed or running over networks, including Java web start applications, are sandboxed by the security manager of the JVM. A sandbox may be thought of as a restricted environment. This means that the instruction set they are allowed to use is severely restricted such that only completely harmless operations are permitted. For these applications to gain more access rights on the local machine (the client machine on which it is deployed or running), the JAR files need to be digitally signed. Again, the Sun tutorial has a detailed explanation about what this involves. Some of the restrictions include:

• Cannot I/O to local files or execute them
• Can only phone home to the host from which it was downloaded – not to other clients or servers
• Warning messages displayed to user

Applets

Applets are another means to run remote code on a client machine. They differ significantly from the Java Webs Start because they do not run as a desktop application, but rather via a browser plug in.

JApplet is a Swing component; however it inherits from neither JComponent nor JFrame. Instead, it inherits from Applet, which is an AWT component.

Deploying an applet on a web page

W3.org’s <object> tag definition | Sun tutorial on Applets

The Sun tutorial and the W3.org specification disagree at the time of writing – the Sun tutorial still is uses the <applet> tag, whereas the W3.org specification says that the <applet> tag has been deprecated in favour of the object tag. The <applet> still tag works on all Java enabled browsers, and it is not too much effort to switch from one to the other – the commonly used attributes are almost identical. In the examples below, the only difference is that the “code” attribute is replaced by the “classid” attribute”.

Using the applet tag:

<applet code="some.package.MyApplet"
   archive="dist/myApp.jar"
   width="400px" height="300px">
      <param name="myString1" value="myValue1" />
      <param name="myDouble2" value="2.0" />
</applet>

Using the object tag:

<object classid="some.package.MyApplet"
   archive="dist/myApp.jar"
   width="400px" height="300px">
      <param name="myString1" value="myValue1" />
      <param name="myDouble2" value="2.0" />
</object>

Java applet code

Before jumping into coding the applet, it is a good idea to think about JApplet’s inheritance hierarchy. Because it inherits from neither JComponent nor JFrame, it will be missing some API (such as setTitle(String), and setSize(int, int)), and behave differently. A word of caution also that JApplet has some interesting painting and layout quirks, thus as a rule of thumb, avoid doing any GUI design work directly inside of an applet – instead do all the GUI design work in a JPanel, and then put the JPanel inside the JApplet.
Another important distinction to make is the entry point into a standard Java application is different from the entry point into a Java applet. A standard Java application always needs (at least) one class that contains the special method with the signature:

public static void main (String[] args)

This is its entry point, and any command line arguments get passed in via the string array. However, in the applet, there is no “main” method, and certainly no array of strings for command line arguments. However, the same effect is achieved using the init() method of the applet, and the parameters passed in using the <param> tags within the <applet> or <object> tags.

public class MyApplet extends JApplet {
   private String myString1;
   private double myDouble2;
   public void init() {
      myString1 = getParameter("myString1");
      myDouble2 = Double.parseDouble(getParameter("myDouble2"));
   }
}

Once this is done, you can do anything with an applet that you can with any other java application. Note that the same security restrictions still apply to code that is deployed over a network – the applets can be sandboxed.

Applets’ external communications

Applets can communicate with other applets, the browser, and any other machine through sockets. However, if the applet is sandboxed, its socket communications will be restricted to only the host machine they originate from.
Applets communicate with other machines through sockets, in the same way that a standard Java application does. See my previous post on socket programming in java for more details.

Applets can communicate with other applets running in the same environment (which in most cases are the other applets in the same web page). They do this through the AppletContext interface. In order to do this, each applet must have a name. This is not a class name, but a unique ID for each applet within the web page. To do this, just add a “name” attribute to each of the <applet> tags or <object> tags.

       Applet receiver = getAppletContext().getApplet("myApplet2");
       if ((receiver != null) && (receiver instanceof MyApplet)) {
           ((MyApplet)receiver).sendMessage(message);
       } else {
           System.err.println("The other applet was not found");
       }

The code above shows how one applet obtains the AppletContext, and uses this to obtain a reference to another applet on the same web page, whose name attribute is “myApplet2″. If such an applet is found, it type casts it to a particular class of Applet (in this case MyApplet), so that it may invoke a method particular to his type of applet – the sendMessage(String) method, which it uses to transmit the contents of the “message” variable. Thus, intra-web page applet communication is achieved.

Applets can also communicate with the browser. This is also achieved via the AppletContext interface. Invoke the AppletContext#showDocument(URL) method to tell the browser to navigate to a new location. Invoke the AppletContext#showStatus(String) method to change the text in the status bar of the browser.

       Applet receiver = getAppletContext().getApplet("myApplet2");
       if ((receiver != null) && (receiver instanceof MyApplet)) {
       ((MyApplet)receiver).sendMessage(message);
       } else {
           System.err.println("The other applet was not found");
       }

Why JDBC?

There are a myriad of database vendors, each with their own software and hardware implementations/ requirements, and their won API: PostgreSQL, mySQL, Oracle, to name a few. In java, there are two main ways to connect to these databases, with Persistence, or with JDBC, and we will be only looking at the latter.

JDBC allows a standard Java application (J2SE) application to communicate with a relational database management system (RDBMS). Like anything else in Java, the programmer uses it by adding a library containing the relevant classes to the class path, and then invoking its API. JDBC acts as a bridge between your java application running from within a Java Virtual Machine (JVM) and a RDBMS running on the native operating system, and provides a pure Java API for database access. The cross platform nature of both Java and JDBC make the Java programmer quite powerful, as he can write the application just once and expect it to work, with minimal need to change the code, for databases from any vendor* and on any platform.

Of course, in order to make access to any database possible through a standard API, the database vendors themselves must conform to a set of standards themselves, and JDBC provides a Service Provider Interface (SPI) that the vendor must implement or conform to. Unless you are a developer for a a database vendor, you will not ever have to worry about JDBC’s SPI. In essence, the Java program talks to the JDBC driver manager, which then translates and relays the message on to the database’s drivers.

Using JDBC, Java programs are able to achieve the classic client-server architecture or three-tier architecture. Also, see my previous post on socket programming in java.

Object Relational Mapping

Objects are the lingua franca of Java programming (and any other OO language). Databases traditionally are about relations – they talk in tables and records. Therefore, unless the database you are talking to is an object database, you will need to do object relational mapping (ORM). That is after reading a record from a table, to transform them into an Object; or prior to writing a record to a table, to construct a record from the Object. Fortunately parallels exist between the object world and the relational world that make these mappings, for the most part**, intuitive.

• Relational <–> Object
• Table <–> Class
• Column <–> Attribute
• Row <–> Instance (of an Object)
• Foreign Key <–> Object reference member variable

Coding with JDBC API

• Plugging your Java program into a JDBC driver – seven quick steps
• Start the database server.
• Add the JDBC’s driver library (usually one or more JAR files) to the class path
• Load the class of the JDBC driver’s entry point using ClassLoader
• Create a statement or a prepared statement
• Execute a query or update via the statement or prepared statement
• Catch and handle SQL Exception
• Finally, close the statement and close the connection

For this demo, we will use the apache derby JDBC driver and Java DB database, since that is developed by Sun and ships with Netbeans, and therefore requires minimal set up and configuration effort.

Setup

To start the database server in Netbeans, go to the services tab, select

Databases --> Java DB --> (right click) --> Start Server.

Easy as pie – this will be more complex if you want to use another vendor’s database. You will of course, need a database instance to connect to, and if you do not already have one, right click on “Java DB” again and select “Create Database”. Give it a name, and create a user name and password to access it with. Note these down, as you will need them later. You will now see an entry that looks similar to:

jdbc:derby://localhost:1527/yourDBname

Note this down as well.

To add the JDBC driver’s library in Netbeans, go to the projects tab, right click on your project and select “Properties”. In the project properties window, select “Libraries”, then “Add Library”. Select the apache derby library, if it already exists, or create a new one (the JAR files are found in the Sun SDK installation directory ./javadb/lib).

Now create a new class for in your project that intends to handle the JDBC access functionality. In its constructor, load the JDBC driver using ClassLoader by doing

Class.forName("org.apache.derby.jdbc.ClientDriver");

Note that we discard the return value of the forName method, because we do not need the actual class. We just want the ClientDriver class to be loaded into the JVM. In addition, we should handle the exception – this exception should never occur unless the JDBC driver’s library has not been added properly.

try {
Class.forName("org.apache.derby.jdbc.ClientDriver");
} catch (ClassNotFoundException cnfe) {
//exception handling code
}

Before we delve into creating SQL queries, we should enable this class to connect to the database.
Define the following static constants for this class, replacing “myDatabase”, “myUname” and “myPword” with the actual values noted down previously.

private static final String DB_URL = "jdbc:derby://localhost:1527/myDatabase";
private static final String DB_USERNAME = "myUname";
private static final String DB_PASSWORD = "myPword";

Now create a method that connect to the database and then returns the connection. This method is intended to be commonly used by several other methods that perform the database access.

    /**
     * Utility method returns a database connection using the specified settings
     * @return Connection to the database
     */
    private Connection getDbConnection() {
        try {
            Connection dbCon = DriverManager.getConnection(DB_URL, DB_USERNAME, DB_PASSWORD);
            return dbCon;
        } catch (SQLException ex) {
            System.err.println("Failed to create connection");
            return null;
        }
    }

CRUD

The actual legwork of this class is to perform create, read, update and delete operations, also known as CRUD. We will implement one of each of the CRUD operations that act on a single object/ single record, as well as just a read operation that returns the entire set of objects/ records.

public void addProperty(Property property) throws SQLException
{
//code goes here
}

public void removeProperty(String id) throws SQLException
{
//code goes here
}

public void editProperty(Property property) throws SQLException
{
//code goes here
}

public Property searchProperty(String id) throws SQLException
{
//code goes here
}

public List
searchAllProperties() throws SQLException
{
//code goes here
}

Create, Update and Delete

Now for each of the operations, we should prepare a statement, execute the query or update, and then close the resources and handle the exceptions.

        Connection dbCon = null;
        PreparedStatement ps = null;
        try {
            dbCon = getDbConnection();
            ps = dbCon.prepareStatement("INSERT INTO property VALUES ( ? , ? , ?)");
            ps.setString(1, property.getPropertyId());
            ps.setString(2, property.getAddress());
            ps.setDouble(3, property.getPrice());

            int affectedRows = ps.executeUpdate();
            if (affectedRows != 1) {
                System.err.println("Failed to update: Number of affected rows = " + affectedRows);
            }
        } finally {
            if (ps != null) {
                ps.close();
            }
            if (dbCon != null) {
                dbCon.close();
            }
        }

• In Java, and in most programming, almost everything is zero-indexed. However, in prepared statements, the first question mark has an index of 1, and we need to keep this in mind.
• In SQL, strings need to be surrounded by quotes. As such, it might be natural to surround the first two question marks above with quote. However, this is not necessary – when using the PreparedStatement#setString(int, String) method, you are already telling the database to expect a string, and thus quotes are made redundant (and cause errors if used).

Next, we execute the update, in this case an Create operation, by invoking PreparedStatement#executeUpdate(). We use the return value, which indicates the number of affected rows, to determine whether the update has been successful.

In this case, we choose to have the exceptions handled by the invoking method or class – that is why each of the CRUD operation methods advertises that they throw SQLExceptions. However, these methods should still close the resources used (the database connection and prepared statement).

Therefore all the statements in the method are put in a try block without a catch block and with a finally block. We close these resources in the finally block, first checking that they are not null as they may never have been initialised, depending on where the exception occurred.

Read

The above template may be trivially modified to work for the Update and Delete operations as well. However, Read operations are fundamentally different; because the flow of information is in the opposite direction than it was in Create, Update and Delete statements. As a result, ORM occurs in the opposite direction, and also, instead of PreparedStatement#executeUpdate(), we will use PreparedStatement#executeQuery().

    public List<Property> searchAllProperties() throws Exception {
        Connection dbCon = null;
        PreparedStatement ps = null;
        try {
            dbCon = getDbConnection();
            ps = dbCon.prepareStatement("SELECT * FROM property");
            List<Property> pptys = new ArrayList<Property>();

            ResultSet rs = ps.executeQuery();
            while (rs.next()) {
                String id = rs.getString("PROPERTY_ID");
                String address = rs.getString("ADDRESS");
                double price = rs.getDouble("PRICE");
                Property ppty = new Property(id, address, price);
                pptys.add(ppty);
            }
            return pptys;
        } finally {
            if (ps != null) {
                ps.close();
            }
            if (dbCon != null) {
                dbCon.close();
            }
        }
    }

We can observe the opposite direction of the ORM by comparing this from the Create method:

ps.setString(1, property.getPropertyId());

String id = rs.getString("PROPERTY_ID");

Notes:
* Provided the vendor is JDBC compliant, and most of the major database vendors are. At time of writing, they number 221. JDBC Drivers provides a full, and searchable, list.

** There are indeed some grey areas, both conceptually and in physical implementation. For example, what happens to static members of classes? What happens to foreign keys that are a combination of multiple primary keys? How are SQL dates mapped to Java date and time objects? There are answers to these, but they are not exactly intuitive.