Organic Programming

I’ve recently received an email  with this content:

hi
i am BLaCkViRuS
Your program Security is Very Low  < ..link to a rapidshare download page>
If you want we can work together and I have anti-cracking programs that you can do
Have a Nice Day !

He was talking about a program we sell at www.obfusc.com. It was not about security but about how strong the licensing is. That link was to a .exe file that was patching obfusc.exe and removed the licensing checks. Nice and friendly email,isn’t it?

In our backlog we have this sprint a story  about introducing  licensing for of our new product (SensErp based on Quark Framework)  and it is a good moment to express my opinion on licensing and cracking:

1. I don’t think that anybody in this world can create the perfect anti-cracking and 100% protection system for a code that will run on a computer where he doesn’t have full control (not having hardware under control)
2. Having people creating cracks for your software it means that your software is good and you have market for it and there are people that love what are you  are doing. I have received many e-mails from customers pleased by using Obfusc Php and that makes me proud and happy (and I make money also, not the salary of a senior programmer or a software manager but enough to consider Obfusc like a decent investment). Obfusc is a good piece of software but the market where it competes is too small for making me a millionaire :)
3. I don’t think that the licensing system for a software should be very strong. It is better for me if somebody really desperate(out of money,etc)  find a way to solve a problem using my software and not using other software or not solving their problem at all.
4. I don’t think that people using such cracks do such a big business damage to a software producer. Somebody is very desperate if will run dubious .exe cracks on his computers at a risk of contacting viruses or other malicious piece of software on his computer. It is like having unprotected sex with a prostitute… some people will do it but they are not very smart  if they take such risks.
5. Some people are greedy,yes,but it is not my problem to solve this issue. Even God looks to have a difficult problem with greedy people We will survive and flourish even if some people are unfairly using our software.
6. People that are creating cracks for software and especially for software created by small companies (like Axiologic) are smart and highly skilled but are not decent persons: they don’t have any morale or ethical justifications for such actions. They will probably pay for it as other people will notice  how they behave.  They will probably loose opportunities for cooperation and they will not even notice that.

What was my answear to  BLaCkViRuS ? Just that: Your offer to help is not acceptable.

Lately I’ve been working on a code editor, based on Flex 4.1, that should be similar to a modern
editor with the so needed syntax highlighting feature. In order to highlight reserved words, strings
and comments, the text is transformed into tokens, which contain useful information for coloring
such as its position in text, length and the type, used to decide which color should be used to
highlight the token.

Everything went well until formatting RichEditableText to highlight the tokens, which was tricky.
My first implementation was something like the code below :

	for (var i:int = 0; i < tokens.length; i++)
	{
		var tlf:TextLayoutFormat = new TextLayoutFormat();
		tlf.color = tokens[i].color;
		richEditableText.setFormatOfRange(tlf,
			tokens[i].beginIndex, tokens[i].endIndex);
	}

This implementation was way too slow: if I pasted a text with around 125 of words to be colored, it would take up to 19 seconds. That was unbelievable, so I started searching about the performance of setFormatOfRange method. By reading some forums and the source code for RichEditableText class, I noticed that this method triggers update on all the elements of textFlow, which happened for each of those 125 words and dropped so much the speed of formatting and displaying the text.

So to solve this issue, all the format operations should be wrapped into a single one to be executed, and when this is done, trigger the update only once.

Fortunately, the Text Layout Framework provides an API for doing just this through flashx.textLayout.operations package. So I wrapped each format operation into an ApplyFormatOperation, which was added to a CompositeOperation object and executed the latter one.
Below is a sample implementation for my improved solution :

	var compositeFormatOperation:CompositeOperation =
		new CompositeOperation();
	for (var i:int = 0; i < tokens.length; i++)
	{
		var operationState:SelectionState =
			new SelectionState(richEditableText.textFlow,
				tokens[i].beginIndex, tokens[i].endIndex);
		var leafFormat:TextLayoutFormat = new TextLayoutFormat();
		leafFormat.color = tokens[i].color;
		var formatOperation:ApplyFormatOperation =
			new ApplyFormatOperation(operationState,leafFormat,null);
		compositeFormatOperation.addOperation(formatOperation);
	}
	var success:Boolean = compositeFormatOperation.doOperation();
	if (success)
	{
		richEditableText.textFlow.flowComposer.updateAllControllers();
	}

Notice that after the composite operation is executed, I update the textFlow only once.
This improved solution dropped the time from 19 seconds to 0.3 seconds, an 63x improvement that makes
my code editor usable again.

I didn’t use an EditManager because it dispatches TextOperationEvent.CHANGE, which I listen only for text changes (insert, paste, cut, delete) on my RichEditableText. So if text changed, I would set color on some tokens by calling applyFormat on EditManager, which would send a new TextOperationEvent.CHANGE and I would ended up in a infinite loop.

I’m sure there might be better ways of doing this (I just haven’t discovered yet), so I’m open to new suggestions and I hope this post would serve as a starting point for others having similar task with the new text controls inside Flex Spark framework.

For more information on Text Layout Framework you may have a look at this PDF file

While thinking this weekend on what we are doing with our startup (Axiologic SaaS) I have just realized a  surprising fact:   having better tools and technologies will mean that the value of code will drop as the code will be cheaper to produce. What it the meaning for businesses? But for professional programmers?

Obviously, source code, like anything else, is worth exactly what one person is willing to pay for it but the first signs of a new reality are already  here:  a lot of  code  for enterprise software is available for free ( open source). It is true that lot of current open source code is quite hard to digest,costly to understand and maintain,etc etc. and that is drastically reduce the value of such code ( but it is not my main point for this post )

For  big software providers,already is true that the value is not in their source code but in their brand,selling capacity, business models and huge amounts of capital that is rolling apparently forever.  Also,I’m not talking in this post about the value of platform’s source code (like Windows, Flash Player,iOS,etc) or about the source code of the tools,  I’m talking about software dedicated to companies (CRMs,ERPs,etc).

When we started our startup we looked carefully towards open source solutions that we could offer to our customers at a reasonable price. We found some open source software with value but it is not all roses. There are some document management solutions there are good,some  open source CRMs  relatively decent (but too complex and hard to sell and deploy,train),  some open source ERPs that could work  but the cost and risks of adapting such solutions is very high  for an self-booting startup.  We already had some experience trying to build RAD frameworks and we have decided to invest our money and effort for create  better technologies that should  dramatically decrease the overall costs of creating and maintaining enterprise source code.   Huge and very risky step, but hey, so is life itself:).

In  enterprise application we can found at least 4 types of code:

• database schema modeling code (code in a DDL)
• business logic code (in rule engines,workflows or OOP code)
• user interface code
• platform/framework code (generic code that doesn’t address directly the end users needs)

My believe is that lows of physics will demand that  in the end all applications that require indefinite level of customization in order to get real and consistent business value  will be based on a very strong and customizable platform and a set of open source code (or anyway low value source code)  that will act like configuration for the platform.

The industry is already going in this direction by putting efforts toward MDA or DDD  (I’ve already mentioned in a previous post). Big ERPs and CRMs names from Microsoft or SAP  have such kind of architecture that put some customization in business rules or workflows but these solutions are not even trying to walk on the highest possible peaks. And it makes probably sense for big companies and their customers but puts walls for small and medium companies that should get the best software for their needs at a reasonable cost. Many managers or owners of small and medium companies already have skills to create Excel spreadsheets to create some proto-application for their needs. We are working for a platform to replace those poor quality applications with fully working, professionally looking and fully customizable business applications at a fraction from current software developmental costs (hopefully almost the same costs like creation of a few spreadsheets).

In the end, a nice picture with one of the highest peaks in Oriental Carpathian mountains (a peak I have just visited this weekend)

Peak Toaca,Ceahlau

Flash Builder 4 comes in 2 flavors: Standard Edition and Premium. With Standard Edition you can’t run unit tests inside Eclipse. If you’re reading this post then I guess you’re looking for an easy and free way of unit testing your flex application. There is a easy way of doing this. All you have to do is create a project, add some swc libraries and set up unit testing.

For a Flex 4.1 environment, you’ll need to add flexunit-core-flex-4.1.0 and flexunit-uilistener-4.1.0 libraries to your project, write test classes and suite class to include the tests, and in the end you should write a little code to Default Application file.

Here is a sample unit testing project you can look over.

The whole post was meant to be short and provide a simple, but reliable way of unit testing in Flex.
For more details and more advanced features you can check out the official website.

One day I came across a task which involved using some sort of collection that was bindable
and could store and retrieve data like a dictionary does. And soon I got a new task : to build
such a data type.

Practically I needed a mix between Dictionary and ArrayCollection, so my first idea was to
extend Proxy, in order to access and retrieve data using square brackets, and after that to
create similar methods to ArrayCollection. But after looking over the source code of ArrayCollection,
I’ve noticed it inherits Proxy, due to ListCollectionView inheritance, which saved me from the classic
‘reinventing the wheel’ and led me to improving the wheel instead.

My plan was to modify the methods of ArrayCollection so it adds/removes links to the added/removed
objects and this way they could be retrieved later.It was pretty straightforward to override the methods
and add/remove references to items,which were stored in a Dictionary. It was also easy because I didn’t have to override each method as some of them are reused. For example, addItem  just calls addItemAt.

Besides this basic functionality, I thought it would be useful to add one more feature : adding and
removing references to items based on given properties. Let’s say you add objects with
quantity, price and label properties, using the plain addItem method and you still want to retrieve
them using square brackets. You can do this, but first you have to decide which properties should be
the keys that references to the object. Below is an example for more clarity.

// here I declare an empty DictionaryCollection and set the label property as a key field
var a:DictionaryCollection = new DictionaryCollection(null,"label");
a.addItem({label: "Bread", price: "2", quantity: "1"});
a.addItem({label: "Butter", price: "3", quantity: "2"});
a.addItem({label: "Tea bags", price: "1", quantity: "4"});
a["Jam"] = {label: "Jam", price: "4", quantity: "1"};
// now I can retrieve each object based on its label
trace(a.getItemByKey("Butter").price);   // 3
trace(a["Tea bags"].quantity);             // 4
trace(a["Jam"].label); 			     //	Jam

I hope this code snippet explains pretty well how a DictionaryCollection can be used and you can download it following the link below. I also included the test class for unit testing.

DictionaryCollection

As a conclusion, I’m happy that reading the source code of Flex saved me hours of work and helped me to build a custom data type that should blend seamlessly with the framework.

The  model is the central part of any MVC application.  But what exactly the models are? Simplistically,one can see the model as  database schemas plus some data structures that reflect data from databases at runtime.

I have tried this “simple” approach in the past frameworks (even in Quark 1.0) and we used classes that mimics the usual OOP classes that languages as Java or Action Script provides by default.  In Quark 2.0 we have created a new XML based, OOP language that we call Quark Schema that is providing support for nice and useful models:

• declaratively describe the persistence and the life of the objects
• declaratively describe the constraints, validation and relations between model’s objects
• describe class behavior
• is fully editable using GUI tools that we are currently building

We found that we can discern 6 categories of objects (they are not orthogonal categories ):

• transient objects – objects that exists only at runtime in one instance of the application and should not be saved in a persistent storage
• globally persistent objects: objects with global identity (even for different users )
• locally persistent objects : objects that have no global identity but have a local identity inside of a global or transient object
• relational objects – objects that mimic the structure of relational tables
• document objects – graph of objects that contain other objects
• shadow objects -objects that doesn’t contain user data but inner information of the system.  Example of shadow objects : configurations and the structure of the view. Shadows get automatically generated and one  is doing customization by using visual tools organically embedded in the application but it doesn’t care much about shadow’s structure.
  
  Observations:
• Global objects that contains other local objects (or are persistence roots for graph of objects as we call it ) should be seen as complex documents structured as graphs that can be used as configurations or data structures. Quark Framework allows you to describe models in the form of normal “relational” objects or in form of  ”document” objects.
• Document objects can be seen as relational objects with fields that are root for trees containing local objects  serialized as XML strings.
• Shadow documents are usually document objects.
• Transient objects can be global or local
• Members of classes can be declared  transient and will not be present in the database schema but their values get serialized depending of their declaration as  local,global( or created as transient objects or no)
• We have discovered that local objects have an interesting property: when they are assigned to a member of a class member from a different persistence root (different persistence global root object) they should be automatically cloned.   Quark Schema is automatically doing just that and we discovered that is nice and useful in real cases.

Apparently all these rules and observations are a bit complex, fortunately the business user will use a simple UI to create such classes (models) and should not worry much about persistence aspects.  It will just communicate his intentions and the system will do “the magic”. Local objects are usually shadow objects and will be a concern for component creators and rarely for the business users (customers or consultants).

This article is long enough so I will not present  any syntax but I want to mention only that for Quark 2.0 we have got all these aspects working and well tested with unit testing.  Drop me an email for any questions.

At the end, take a look at this picture that is saying: “the model should be fat but it should be constructed with skinny,elegant and nicely shaped classes”:

Creating  above picture piece by piece is  a laborious process. For me, it took 1 minute to create using Google search for images and a screenshot. Similarly, our goal is to create a tool that  will allow  creation of a working business application in minutes and we believe that we are not far from it! A new direction I’m contemplating now is to have  Quark Framework  as a tool for creation of mashup applications [1] with data from multiple business applications. It is a logical use of Quark and fortunately it results from our design,without much supplementary programming work.

References:

[1] http://en.wikipedia.org/wiki/Mashup_(web_application_hybrid)

We are happy to make a public release of Quark Framework 1.0.

For the last couple of  months we have been working on Quark 2.0 and we’ve decided to have Quark Framework 1.0 released under LGPL license.

Quark RAD Framework comprises two key components – a core library written in Java  (which provides the basic server side persistence and authentication  services that are required by most applications) and a set of flex classes so developers can easily use to rapidly engineer UI of the applications.

As we are working for 2.0, we discontinued support for 1.0 (except fixing things required for applications we build with 1.0) On the other hand, if you are a good Java and Flex hacker you could find good piece of code to reuse in your applications.

Documentation for 1.0 is scarce as we focus on explaining and documenting 2.0.

Download Source |  Binary

In the quest for eliminating all unnecessary barriers in having ultra rapid application development we realized that the save button is sometimes quite Evil. For web applications the Save button is somehow inefficient and could lead to confusion and loose of data because of browser crushes,network errors,user mistakes. The save button is also increasing the complexity of the system. Many hardcore programmers will say that it is not much but our framework wants to allow even non-programmers to create working applications with an effort equivalent with that of creating a spreadsheet.  We think that the approach of periodic auto-saving is the way to go in Quark for  almost all cases.

If the framework provides cheap (or free)  undo,redo operations for all operations that users can do,naturally that we can have nice applications without the “save button”.

What “save button” provides and “workarounds” could be required:

• the biggest issue: workflow and integration with process management capabilities. If a user is just playing with data in UI,could trigger workflow changes unintentionally. Business application that have workflows are normally used by serios people that will not just click randomly and press keys,but still, this could be a problem.
• protection for “cat” operations – if a user is just playing with his keyboard or a cat is walking on it,bad data could find a way to the database
• implementation of atomic changes:provide consistent persistent storage.  In some cases,changes in a part of an object without changing other parts will lead to an invalid states of the object. This problem will be avoided with a validation, Quark Framework will not auto-save invalid objects.
• implementing transactional operation for graph of objects will require additional,a bit more complex validation

I think that cases that will require a higher level of validation,could have a standard “validate change” button or “apply” button in the UI and the “auto-save” will not save those objects that are not manually validated.

Quark framework 2.0  provides for free (almost) undo,redo and auto-save. These features were not present in Quark 1.0.   So,what do you think?  Do you really need to put users pressing the “save button” after each important change ?

Nothing is wrong! MVC is the most famous design pattern after singleton but people have very diverse opinions and there are quite a lot of misunderstandings about this powerful design pattern.

MVC design pattern is  illustrated by this figure (from wikipedia [1] ):

Even if those dashed and solid lines are looking a bit dubious, in many existing MVC implementations  the reality is even more complex than this, look at the next  image from [2]:

Simple? No? Necessarly complex? There is no final answer yet, but in Quark 2.0 we have a simpler and elegant communication  paradigm between Models,View and Controllers.

In Quark, the controller is controlling the model and the view is strictly rendering the Model while triggering the controller when the user is asking for something. Also, as you can see in the picture, we are fans of fat models [3] ,  skinny controllers and dummy,auto-generated views like in Naked Objects , MDA,DDD [4] .

We know that some programmers do not have the capacity to understand that a “Model” is not just a wrapper of a database table but our approach that we call “Quark Schema”   should make things easier for anybody: advanced programmer or clueless user. Even non professional programmers could do the right things without worrying much about philosophical and technical details of MVC.  Our goal is to make a tool and a programming framework that will look and be simple like doing things in  Excel but resulting in nicely growed Organic Programs.

References:

[1]  MVC in Wikipedia: http://en.wikipedia.org/wiki/Model–view–controller

[2] Pure MVC page on Wikipedia http://en.wikipedia.org/wiki/PureMVC

[3] Links about Fat and skinny Models and controllers, anemic domain model,etc

http://www.martinfowler.com/bliki/AnemicDomainModel.html

http://nuts-and-bolts-of-cakephp.com/2009/01/06/another-way-to-think-about-mvc/

[4] Naked Objects, About MDA and DDD : http://en.wikipedia.org/wiki/Naked_objects

And just for fun (and profit for some): About Fat Models

Many people believe  that programming is like building of houses and programmers are architects. Others, believe that programming should be more like farming and growing of beings.
A perfect explanation for this thinking is expressed in [1]: “Systems grow and evolve. Often programs die, or succumb to disease. Yet, at the same time, I can feel the pulse of life as information permeates through the network. I no longer seek to claim to be a builder, but one who plants seeds and cultivates the soil. The biological metaphor may sound humble when contrasted to the titles of architect or engineer, yet for a system to bear fruit, it will take cultivation.”
Different approaches (but no one being mainstream) tried to integrate programming with NLP or with multi-agent system, but our current view about Organic Programming can be resumed for a set of simple principles:

1. OP should lead to high quality programs,with reduced costs of developing and adapting programs to the needs of the final user.
2. following of the best programming practices while hiding complexity for the developer by using advanced frameworks and good UI
3. tight integration of the tools used for developing the program with the program itself. These tools can be used at runtime to customise or even to create new features.
4. use the right language technology for the right thing. A wider adoption of technologies like rule engines,workflows and well designed domain languages could improve the quality of the programs.

Everybody wants to achieve the good effects of first principle of OP, but few succeeds because they are building programs instead of growing programs. Or at least this is what we believe.
Current mainstream frameworks like Rails or Java framework are not respecting the first principle because they are too focused on code and are using a technology for building frameworks (OOP) for creating programs.
The 3th principles is respected only in some programs, a notable case are spreadsheets and in the next articles we will insist on how our technology called Quark Framework goes towards respecting this fundamental principle of OP.  A second version of the Quark framework will be released soon. The first version will be published as open source software next week.  At least in the near future we will focus only on organic programming for enterprise software [3] and not on organic computing [2].

References:
[1]   http://descmath.com/data/organic.html

[2]   http://en.wikipedia.org/wiki/Organic_computing

[3]  http://en.wikipedia.org/wiki/Enterprise_software