Showing posts with label programming. Show all posts
Showing posts with label programming. Show all posts

Thursday, February 9, 2012

python | head == IOError: [Errno 32] Broken pipe

I spent way more time today than I could afford fighting against a python script that, when piped to head, generated the following message:

IOError: [Errno 32] Broken pipe

Now had this been a simple program I could have just caught the IOError and managed, but that didn't work. The program was using Popen from the subprocess module and and when I "handled" the exception I got a weird exception:


tail: stdout: Broken pipe
close failed in file object destructor:
Error in sys.excepthook: 
Original exception was:

Note, the original exception was empty...

I am a Python neophyte, and of course it took me a long time to figure out how to properly handle the broken pipe. The meat of the matter is that Python does not ignore SIGPIPE, it creates an exception for it. My googling and stackoverflowing lead me to more exception handling including replacing the sys.excepthook and other hackery.

The solution was of course much simpler and eventually found on the comp.lang.python archive. I needed to restore the signal handler for SIGPIPE doing the following:


from signal import signal, SIGPIPE, SIG_DFL 
#Ignore SIG_PIPE and don't throw exceptions on it... (http://docs.python.org/library/signal.html)
signal(SIGPIPE,SIG_DFL) 
for x in range(100000):
        print x
The Python manual was a bit misleading the way that I parsed the following:

Python installs a small number of signal handlers by default: SIGPIPE is ignored (so write errors on pipes and sockets can be reported as ordinary Python exceptions) and SIGINT is translated into a KeyboardInterrupt exception. All of these can be overridden.

What seems to really happen is that SIGPIPE is translated to an IOError, at least in Python 2.6.1, and in order to make it not you need to use the SIG_DFL one.  Note that SIG_IGN doesn't work.





Sunday, September 28, 2008

Portlet Factory Java Expressions

I found something in Portlet Factory that I have wondered about for a few months now. When you pick an item from the Java category of the reference/value picker there are some useful values for the java expressions. USERID is one I use all the time. I have been typing the ${Java/Boolean.TRUE} reference quite a few times lately, and I really wanted to add it and others to the list.

In order to add to the list you need to add items to the WebContent/WEB-INF/config/IndirectJavaExpressions.config file. It is a properties file format, and therefore pretty easy to figure out.

Here is what I added:
  • TRUE=Boolean.TRUE
  • FALSE=Boolean.FALSE
  • NULL=null

I use those almost every time I use the tool. I am sure I will figure out more as I go. Now I just need to remember to take .cvsignore away for that file.

Sunday, June 29, 2008

WebSphere Portlet Factory -- Introductions

I have been working in WebSphere Portlet Factory lately and I am beginning to get a feel for how it can make web development easier, especially when building portlet applications. Learning to use it involves a different way of thinking about building web applications. The process of building applications using it initially warped my mind the same way that learning LISP did after many years of C.

I have yet to find a good series of tutorials that teach portlet factory using the "hello world" style. Two starting points for the newbie are the wiki and the redbook. The wiki has some good content, but not a good starting from scratch set of tutorials. The redbook does do starting from scratch, but in a grandiose way that forces you to ignore errors that are displayed until you finish things later... It is a mammoth work, some 730 pages in all. It is a little intimidating when you are just starting out.

Another recommended area to get support is the forums (9 in all). The 4 most relevant to the beginner are:
The problem is that there are 9 of them and 4 are useful, and you are never really sure where to search for the answer to your question. I haven't found a way to search across only these 4 forums and frequently have to search across everything at IBM, which is not exactly helpful...

One thing to know is that the tool is far from free. If you own WebSphere Portal (6 and above) you are entitled to one seat of the designer, and Eclipse plugin that allows you to build portlet factory applications. You are also entitled to run those applications inside of the WebSphere Portal environment. Things get more complicated if you want to run outside of WebSphere Portal (more on that another day), and the pricing for the designer standalone is similar to that of IBM's Rational Application Developer product.

That being said, I think that the cost might be justified when stacked against the complexity of building applications in the J2EE, even with the support of frameworks like Spring. It really is remarkable how quickly you can put a web application together that displays data or does CRUD.

You can download the Portlet Factory 6.1 Beta from the WebSphere Portal 6.1 Beta Site.

If I can find my blog legs I will be doing more on this topic, and may even take a shot at the "hello world" style tutorials.

Sunday, December 31, 2006

Getting into the (Spring Web) Flow

I spend my days doing web application development in Struts, and lately have been spending more and more time developing portlet applications for WebSphere Portal. I recently been using a new framework that eases some of the problems that I have recently had using Struts to build flow based web and portal applications.

Problem 1
One of the things I consistently find difficult to do in Struts is restricting a user's navigation to a well defined path, especially when users start tampering with URLs. In order to ensure that we are in a valid state we:
  1. Use subclassing of actions, where the parent action defines an abstract execute method for the child to implement and checks state for us before delegating to the child to handle the request. This usually leads to very tight coupling, and a headache when something needs to be changed.
  2. We duplicate code in each action to make sure we are in the correct state. Maybe we use helpers, but the code still smells.
Problem 2
Lately, I have been working with Struts newbies. I have seen that the separation between action mappings in the Struts configuration file and the action calls embedded in the views makes it difficult to comprehend the "flow" of an application. To see what is happening you have to process all of the views, looking for forms, links and javascript looking for submissions to action url mappings in the struts config file.

Problem 3
As I try to bring Struts to bear in portal development I have found it even harder to keep things coherent (IMHO, Struts has more than a bit of an impedance mismatch with portal development). A consultant that I worked with introduced me to the portlet state pattern, which is a simple solution that works well for portlet applications. The general idea is that navigation is controlled by the state that the application is currently in, the current state is stored in the session, and actions modify the current state. The controller can ensure that only actions which the current state allows are executed. One shortcoming is that the transitions are encoded in separate Java classes, meaning that the documentation value that a Struts config file would have provided is gone. Given the time I might have come up with an externalized configuration solution but I never quite got around to it.

A better way?
In building the next generation of a login/registration/self-help front end for an existing portal I quickly realized that I didn't want to write bunches of code to make sure a user had not changed a url and navigated past part of the login or registration flow (see Problem 1). What I wanted was the portlet state pattern for a Struts web application. I looked at solutions for Struts and found a few:
Then I found Spring Web Flow.

In less than an hour I had the existing main flow for the application defined with stub pages, and was adding the new states. I ended up breaking some of it down later into sub-flows, as well as playing around with the different types of states more than I probably should have, but it was fun! What I found after a day of work was a well defined, mature, testable, self-documenting framework for defining and enforcing flow in web applications. There are certainly non-flow based web applications out there, but I do know that the vast majority of the web applications that I have written in my life need some way of enforcing a known set of transitions between states.

One of the things that surprised me was how easy it was to "get something working" with mocked up pages and "fake" actions but the navigation that I would expect between pages, and then drop in real actions/data/views one at a time, having a "working" application at each step of the way.

SWF is that it isn't coupled to a web application framework (to be fair Commons Chain isn't either). This means that it should work great when I am wearing my portlet development hat, and that it is testable without needing to mock up a servlet environment. To accomplish this SWF implements an abstraction layer over the Servlet API. This means that it can be tested outside of a servlet container, and that it can work with other APIs (like the portlet API). In reality it is an engine that other tools and frameworks call into. For more information look at the documentation for ExternalContext. It works with Struts, Spring MVC and JSF out of the box.

For those "afraid" of XML configuration, there is a fully functional programatic API for building your flow. I found the XML configuration more practical, and it has the benefit of being able to be manipulated using XSLT to generate documentation for the flow (that my boss, and the customer, could read). I even hacked together an XSLT template to generate a dot file that can be processed by graphviz to generate a visual representation of the flow (This won't be necessary when the Spring IDE WebFlow Editor is complete, hopefully someday soon).

A couple of stumbling blocks I hit along the way:
  1. Many of the tutorials that are out in the wild are for a pre-1.0 release of the framework and therefore might not be a 1:1 match to the current version. The project has been around for quite a while (some history). The API, and configuration syntax has changed for the 1.0 release and I did hit a couple of snags trying to follow the examples that I found out on the net. There is a whole set of examples bundled with the framework that were a good introduction. The manual for the framework is a good technical reference, but is far to low level. One thing I think that the project needs is a good quick start tutorial that is up to date with the 1.0 API.
  2. The configuration uses OGNL to define expressions which are used to retrieve values from the contexts and pass to underlying objects. The error messages that are generated when the OGNL expression can't be parsed or bound are sometimes "less than helpful". I don't know if more work on error handling from the parser would help in this case, or if there is not enough contextual information being passed back by the parser.
The SWF framework addresses several of the weaknesses of using Struts to build flow based applications. It provides the ability to force a navigation path, has a single point of reference for the possible states and transitions that a flow can be in, and works across web frameworks. It even supports portal development out of the box.

I will be using Spring Web Flow more in the near future, and can't wait to get to try it on some of the more complicated portlets that I have been working on. I know it would have made the last search/display/edit/create portlet that I did a lot easier. I wonder if I should rewrite it using SWF...

Note: One interesting project that looks similar to SWF is Commons SCXML, which is based on a W3C specification. The format of the configuration files is similar to the SWF configuration file, and it will be interesting to see how the two technologies grow with/against one another.


Links: