Ayende on Dependency Injection

Here is another post from Ayende, this time on rejecting Dependency Injection inversion. It is worth reading and highly recommended for those, who don’t like the very idea of IoC usage. For example, Alex Ilyin is from that camp. Recently we had quite interesting conversation on this topic in comments to this post.

Updates on the development process

The beginning of 2010 year was quite busy for the entire DataObjects.Net 4 team. One part of it headed with Alex Yakunin, has been busy with the late DataObjects.Net 4.1 release (which in fact still haven’t been shipped), the other has been working hard on the upcoming DataObjects.Net 4.2 release.

While other developers from the latter part have been working on full-text search, object-to-object mapper and other features from 4.2, personally I was working on highly demanded by customers SQL Server Compact Edition 3.5 support. The work is already done, CE support related code was committed and is being tested by our CI server during the last 2 days. The feature is going to be included into 4.2 release. As for Sql Server CE, it has a huge number of restrictions and I’m going to list all of them in one of the future posts. To sum up, it can replace its big brother (Sql Server 2005/2008) only partially.

Speaking about the upcoming DO.Net 4.2 release, we are planning to make a stable Release Candidate at the end of the January (at the beginning of the February) and deliver DO.Net 4.2 final in the middle of the February, so stay tuned.

As for me, I’ve finished working on 4.2 branch and there is a high possibility that I’ll be busy during the next 2 or 3 months with another project, not connected with DataObjects.Net development at all. The project is going to be very complicated and, as it usually happens, has very limited time-frame, but I hope that my participation will help a lot/a bit. My role will be mainly as business analyst/system architecture. I’m preparing myself for working on endless Word documents, requirements, UML diagrams, precedents, etc. So wish me good luck. =)

However, if I have enough time I’ll keep on posting some regular notes on DO.Net development process.

A frank confession

Alex Ilyin lifts the curtain on terrifying details of DataObjects.Net 4 development process.

Query preprocessors, Inversion of control & Localization support

LINQ translator extension

As I promised earlier, we were going to make LINQ translator extendable and finally we’ve made this. The extension mechanism is called “Custom LINQ query preprocessors” and is already included into DataObjects.Net 4.1 code base.

Custom LINQ preprocessors must implement public interface IQueryPreProcessor which is defined in Xtensive.Storage assembly. Here it is:

public interface IQueryPreProcessor
{
  Expression Apply(Expression query);
}

As you might see, the contract is quite simple and straightforward: your preprocessor receives the whole query, modifies it in  the way you need, and returns the modified one. All preprocessors are called before the query is processed by internal LINQ translator, so it is the right time and place to apply necessary modifications.

Connecting preprocessors to translator (IoC)

After you have written you preprocessors, it is time to plug-in them to DataObjects.Net. This is done with the help of Inversion of Control concept. In order to follow it, you need to take the following steps:

1. Add reference to Microsoft.Practices.ServiceLocation.dll assembly. It is shipped with DataObjects.Net 4 and can be found in %DataObjects.Net Directory%\Lib\CommonServiceLocator directory.

2. Configure IoC container through application configuration file.

Add this line to configSections part:

    <section name="Services" type="Xtensive.Core.IoC.Configuration.ConfigurationSection, Xtensive.Core"/>

Add the corresponding configuration section:

<Services>
  <containers>
    <container name="domain">
      <types>
        <type type="Xtensive.Storage.IQueryPreProcessor, Xtensive.Storage" mapTo="Xtensive.Storage.Samples.Localization.QueryPreProcessor, Xtensive.Storage.Samples.Localization" singleton="true" />
      </types>
    </container>
  </containers>
</Services>

Note the usage of named service container (“domain”), the  IQueryPreProcessor type as an interface of a service and how it is mapped to the concrete implementation.

3. The last step is to configure Domain object & the above-mentioned service container.

// Building domain
domain = Domain.Build(DomainConfiguration.Load("Default"));

// Configuring domain-level services
var configurationSection = (ConfigurationSection)ConfigurationManager.GetSection("Services");
var container = new ServiceContainer();
container.Configure(configurationSection.Containers["domain"]);
domain.Services.SetLocatorProvider(() => new ServiceLocatorAdapter(container));

LINQ preprocessor in action

Having these actions done, we get the capability of using non-persistent localizable properties (Domain model can be found here) in LINQ queries:

using (var ts = Transaction.Open()) {

  Console.WriteLine("Implicit join through preprocessor");
  var pages = from p in Storage.Query.All<Page>()
  where p.Title=="Welcome!"
  select p;
  Console.WriteLine(pages.ToList().Count);

  ts.Complete();
}

Pay attention that neither PageLocalization type nor its members participate in the query, original p.Title expression in Where clause is used instead. As we know, Page.Title is not a persistent property and regular LINQ translator doesn’t know how to translate this expression. But having the initial query preprocessed with Xtensive.Storage.Samples.Localization.QueryPreProcessor makes such kind of expressions possible to use. The only thing the preprocessor makes is the replacement of p.Title expression to something like this:

p.Localizations.Where(localization => localization.CultureName==LocalizationContext.Current.CultureName)
          .Select(localization => (string)localization[“Title”])
          .FirstOrDefault();

That’s it.

The source code is available in our public repository in Xtensive.Storage.Samples.Localization folder.

Happy preprocessing! =)

Logging, part 3. Configuring logging through log4net

In the previous post I demonstrated how to configure and use internal DataObjects.Net logging capabilities. These are rather useful and flexible but in case you want much more flexibility or something, using one of the external logging frameworks could be the right choice for you.

As I already mentioned, external logging components are connected to DataObjects.Net through the mechanism of adapters: DataObjects.Net => adapter for logger => logger.

In case of log4net you’ll need the following assemblies:

• log4net.dll (can be found here)
• Xtensive.Adapters.log4net.dll (is contained in DataObjects.Net installer)

The next step is to configure both DataObjects.Net & log4net.

Add these sections to configSection block of your application configuration file:

    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler,log4net"/>
    <section name="Xtensive.Core.IoC" type="Xtensive.Core.IoC.Configuration.ConfigurationSection, Xtensive.Core"/>

Xtensive.Core.IoC namespace goes for basic Inversion of Control implementation however it is powerful enough to accomplish most of appropriate tasks. In this case it is used to map Xtensive.Core.Diagnostics.ILogProvider interface to some external implementation (Xtensive.Adapters.log4net.LogProviderImplementation type).

<Xtensive.Core.IoC>
  <containers>
    <container>
      <types>
        <type type="Xtensive.Core.Diagnostics.ILogProvider, Xtensive.Core" mapTo="Xtensive.Adapters.log4net.LogProviderImplementation, Xtensive.Adapters.log4net" singleton="true"/>
      </types>
    </container>
  </containers>
</Xtensive.Core.IoC>

The last step is log4net configuration:

<log4net>
  <appender name="FileAppender" type="log4net.Appender.FileAppender">
    <file value="log-file.txt" />
    <appendToFile value="true" />
    <lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date %-5level %logger - %message%newline" />
    </layout>
  </appender>
  <root>
    <level value="WARN" />
    <appender-ref ref="FileAppender" />
  </root>
  <!-- To log warnings & errors from Xtensive.Storage.* loggers --> 
  <logger name="Storage" additivity="false">
    <level value="WARN" />
    <appender-ref ref="FileAppender" />
  </logger>
  <!-- To log all SQL statements -->
  <logger name="Storage.Providers.Sql" additivity="false">
    <level value="ALL" />
    <appender-ref ref="FileAppender" />
  </logger>
</log4net>

Having these configuration steps done, you’ll get DataObjects.Net & log4net bundle configured & working.

Logging, part 2. Architecture & configuration

The main goal was: how to make logging and its configuration easy for simple scenarios and in the meantime highly adaptable for complex ones.

In order to achieve the required level of flexibility most logging frameworks have the following components:

• Loggers (named instances of some public class or interface (usually ILog) that provides developers with functionality to write diagnostic messages to).
• Appenders (output destinations of above-mentioned loggers).
• Log manager or log provider (usually a central access point of a framework. It resolves loggers by their names).

DataObjects.Net logging system follows exactly this pattern but bearing in mind that it must provide the possibility to plug-in any logging framework it introduces its own set of abstract components (actually loggers & log manager, but not appenders) which in fact just wrap up the plugged-in ones and simply redirect diagnostic messages to them. Moreover, in case when none of standalone logging frameworks is plugged-in, DataObjects.Net contains its own simple implementation of those components.

The main access point is the public static LogProvider class with one method LogProvider.GetLog(string logName), which is used to resolve a required ILog instance by its name. Once instance of ILog is obtained, it can be used to log Debug, Info, Warning, Error & FatalError messages through the corresponding methods.

It could be considered as a good practice when members from one namespace log their messages into the same logging space. This namespace-based approach is quite useful as usually a namespace contains a set of classes that are closely coupled and execute some shared piece of programming logic, therefore the idea to merge their diagnostic output in one log seems to be rather sensible. Due to this approach DataObjects.Net contains a set of predefined loggers for most frequently used namespaces, such as: Xtensive.Core, Xtensive.Storage, Xtensive.Storage.Building and so on. Each of these loggers has name which corresponds to its namespace except "Xtensive." prefix. Say, logger for Xtensive.Core namespace is named as "Core".

For usability reasons the above-mentioned namespaces contain public static class named Log which exposes the same set of logging methods as ILog interface. As you might understand, each of these static Log classes is no more than a connector between its consumers (classes which use it as a logger) and corresponding ILog instance that is transparently constructed on demand.

Configuring internal DataObjects.Net's log output

Internal logging subsystem is not as powerful as some well-known logging monsters but rather flexible and doesn't require any additional components. Configuration of DataObjects.Net's logging is made in application configuration file (app.config or web.config).

First of all, include Xtensive.Core.Diagnostics section into configSections section:

  <configSections>
    <section name="Xtensive.Core.Diagnostics" type="Xtensive.Core.Diagnostics.Configuration.ConfigurationSection, Xtensive.Core" />

The second step is to configure logs (appenders in terms of log4net):

<Xtensive.Core.Diagnostics>
  <logs>
    <!-- Use these settings for Xtensive.Storage.* logs -->
    <log name="Storage" events="Warning,Error,FatalError" provider="File" fileName="Storage.log" />
  </logs>
</Xtensive.Core.Diagnostics>

Note that each log has a name which is equal to the namespace where it is located except "Xtensive." prefix. This is true for logs from DataObjects.Net only and might not be true for logs from your own application.

Types of events: Debug, Info, Warning, Error, FatalError.

Types of providers: File (you need to provide file name as well), Debug, Console, Null (no logs at all, analogue of /dev/null), Error.

The example of log:

2009-12-17 00:00:02,052 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Creating connection 'sqlserver://*****'.
2009-12-17 00:00:02,052 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Opening connection 'sqlserver://*****'.
2009-12-17 00:00:02,052 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Beginning transaction @ ReadCommitted.
2009-12-17 00:00:02,068 DEBUG Storage.Providers.Sql - Session 'Default, #9'. SQL batch: 
SELECT [a].[Id], [a].[TypeId], [a].[Name], [a].[Code], [a].[Description], [a].[LongDescription],
 [a].[IsForChildren], [a].[BasePrice], [a].[Price], [a].[SizeString], [a].[HasNoInnerCover]
 FROM [dbo].[Product] [a] ORDER BY [a].[Id] ASC
2009-12-17 00:00:02,068 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Commit transaction.
2009-12-17 00:00:02,068 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Closing connection 'sqlserver://*****'.

Looks pretty good, right?

In the next post I’ll describe how to use external logging framework with DataObjects.Net.

Logging, part 1. Introduction

In the next posts I’m going to describe how logging in DataObjects.Net is designed, how it works and how to configure and use it in most effective way. In the meantime, I’m writing exactly the same chapter in the manual, so this work will be paralleled, although I suppose that the blog version will be a bit more informal than manual’s one.

Let’s start then.

In general, logging is the feature most of software engineers use to track how the system works and analyze when it starts to behave in improper manner. It goes without saying that logging capabilities are essential for any product designed primarily for developers and software companies. But having decided that your framework must use some kind of logging, you immediately face up to another challenge: which logging system to use as there are plenty of them (log4net, NLog, etc.). Moreover, you might want to invent your own super-duper logging system.

This choice is rather simple for small products or libraries: they just use one of the most famous and simple in usage, i.e. log4net, or writes everything to some place that can be set up somewhere in configuration file, i.e. "C:\Debug\". But is this straightforward approach good enough for their customers that use these small standalone components to build something more complex and non-trivial?

In such cases the right word is "Transparent integration", it really matters how your small library can be integrated into large system, is its logging subsystem flexible enough to be easily integrated with logging framework that is used there? These are the questions DataObjects.Net development team was thinking about when logging subsystem was about to be implemented.

To be continued…

BTW, we are going to publish the updated localization sample with the generalized LINQ pre-processor that is used to automatically and transparently join localizable & localization entities and substitute calls to localizable properties soon. Stay tuned!