tomas elfving : IT architect

Performance counters for Sharepoint 2010 WFEs

2012-01-01T22:17:00.000+01:00

Hi, this is a set of performance counters covering both general performance and counters for how Output & ASP.NET cacheing works (providing you have them enabled) including problem description an possible resolution options.

Objects and Counters

Problem

Resolution Options

Processor

% Processor Time (_Total)

Over 75-85%

Upgrade processor
Increase number of processors
Add additional server(s)

% Processor Time - Process (w3wp)

% Processor Time - Process (OWSTIMER)

Disk

Avg. Disk Queue Length

Gradually increasing, system not in a steady state and queue is backing up

Increase number or speed of disks

Change array configuration to stripe

Move some data to an alternative server

% Idle Time

Greater than 90%

Increase number of disks

Move data to an alternative disk or server

% Free Space

Less than 30%

Increase number of disks

Move data to an alternative disk or server

Memory

Available Mbytes

Less than 2GB on a Web server.

Add memory.

Note:

SQL server available memory will be low, by design, and does not always indicate a problem.

Cache Faults/sec

Greater than 1

Add memory

Increase cache speed or size if possible

Move data to an alternative disk or server

Pages/sec

Greater than 10

Add memory

Paging File

% Used and % Used Peak

The server paging file, sometimes called the swap file, holds "virtual" memory addresses on disk. Page faults occur when a process has to stop and wait while required "virtual" resources are retrieved from disk into memory. These will be more frequent if the physical memory is inadequate.

Add memory

NIC

Total Bytes/sec

Over 40-50% of network capacity. This is the rate at which data is sent and received via the network interface card.

Investigate further by monitoring Bytes received/sec and Bytes Sent/sec.

Reassess network interface card speed

Check number, size, and usage of memory buffers

Process

Working Set

Greater than 80% of total memory

Add memory

% Processor Time

Over 75-85%.

Increase number of processors

Redistribute workload to additional servers

ASP.NET

Errors/sec

The number of errors per second that occur during the execution of HTTP requests, including any parser, compilation, or run-time errors

Application Pool Recycles

Should be as low as possible, 0 or close to.Several per day, causing intermittent slowness.

Make sure that you have not implemented settings that automatically recycle the application pool unnecessarily throughout the day.

Worker Process Restarts

The number of times a worker process has been restarted on the server computer. A worker process can be restarted if it fails unexpectedly or when it is intentionally recycled. If this counter increases unexpectedly, you should investigate as soon as possible.

Requests Queued

Hundreds or thousands of requests queued.

Implement additional Web servers

The default maximum for this counter is 5,000, and you can change this setting in the Machine.config file

Request Wait Time

As the number of wait events increases, users will experience degraded page rendering performance.

Implement additional Web servers

Requests Rejected

Greater than 0

Implement additional Web servers

Application Restarts

Should be as low as possible, 0 or close to.Several per day, causing intermittent slowness.

Cache Total Entries

The total number of entries in the cache.

Cache Total Hits

The total number of hits from the cache.

Cache Total Misses

The number of failed cache requests per application.

Cache Total Hit Ratio

The ratio of hits to misses for the cache.

Cache Total Turnover Rate

The number of additions and removals to the cache per second, which is useful in helping to determine how effectively the cache is being used. If the turnover rate is high, the cache is not being used efficiently

Output Cache Entries

The total number of entries in the output cache.

Output Cache Hits

The total number of requests serviced from the output cache.

Output Cache Misses

The number of failed output-cache requests per application.

Output Cache Hit Ratio

The percentage of total requests serviced from the output cache.

Output Cache Turnover Rate

The number of additions and removals to the output cache per second. If the turnover rate is high, the cache is not being used effectively.

Session SQL Server Connections Total

The total number of session-state connections made to the Microsoft SQL Server database in which session-state data is stored.

Database scalability tips

2011-09-21T23:15:00.000+02:00

Avoid Object Relational Mappers. They create complex queries that hard to optimize.
Analyze locks. Row level locking is better than table level locking. Use async replication
Single database choke point? Create parallel databases and let a driver select between them.
Use metrics. Visualize what's happening to your system using one of the many monitoring packages.
Lack of Feature Flags. Turn off features via a flag so when a spike hits less important features can be turned off to reduce load.

WS consumer performance tips

2011-09-16T22:37:00.000+02:00

WS call are always expensive with the overhead each call require. Caching is a way to avoid unneccesary WS calls and improve performance:

The general rules for cacheing are:

• Always cache WS data which is not very dynamic in nature, bu at the same time make sure you don't get functional error by caching data that needs to be fetched fresh.

• Use the ASP.NET cache with a timeout instead of some custom cache like hash table

• Invalidate cache based on business rules or time-out

• Consumers of Web services have the option of calling web services asynchronously or synchronously. Your code should call a Web service asynchronously only when you want to avoid blocking the client while a web service call is in progress. If you are not careful, you can use a greater number of worker and I/O threads cause by asynchronous WS calls, which negatively affects performance. It is also slower to call a service asynchronously; therefore, you should avoid doing so unless your client application needs to do something else while the service is invoked.

© Copyright 2011, Tomas Elfving

WS design for improved performance

2011-08-16T00:07:00.001+02:00

In my experience, these three are the most commonly seen performance-related design mistakes in web service development:
1. Chatty calls.
Network round trips to and from a Web service can be expensive. This issue is magnified if clients need to issue multiple requests to a Web service to complete a single logical operation. The web services should be designed to maximize the amount of work performed with each request. Make sure that the functionality you are offering is worth the time and effort the client had to go to in order to get the request to you in the first place. For better performance, consider combining smaller requests into a larger single request.

2. Serialization of large amounts of data.
Serializing large amounts of data and passing it to and from Web services can cause performance-related issues, including network congestion and excessive memory and processor overhead. Consider redesign.

3. Inefficient state management.
Inefficient state management design in Web services can lead to scalability bottlenecks because the server becomes overloaded with state information that it must maintain on a per-user basis. Common pitfalls for Web services state management include using stateful Web methods, using cookie container–based state management, and choosing an inappropriate state store. The most scalable Web services maintain no state.

© Copyright 2011, Tomas Elfving

Windows Azure Tools extend Visual Studio 2010 out now

2011-08-03T23:02:00.000+02:00

Windows Azure Tools for Microsoft Visual Studio extend Visual Studio 2010 to enable the creation, configuration, building, debugging, running, packaging and deployment of scalable web applications and services on Windows Azure.
Find it on http://goo.gl/fb/kHYZ0, install using the Web Platform Installer.

© Copyright 2011, Tomas Elfving

Commerce Server 2009 and Sharepoint 2007 Evaluation VPC

2010-04-04T22:29:00.000+02:00

On Microsoft Connect, there is a Commerce Server 2009 Evaluation VPC (February 2010 version) available for download to partners and customers. This VPC has Commerce Server 2009 and Microsoft Office SharePoint® Server 2007 (MOSS) fully installed and configured (Core Systems, Multi-Channel Commerce Foundation, SharePoint Commerce Services), including the new Default site with the Template Pack running in SharePoint using the 30 new Web Parts.

Register to download the latest VPC update, complete with a demonstration script, sales collateral, samples, and videos at http://go.microsoft.com/fwlink/?LinkId=164446

© Copyright 2010, Tomas Elfving

Commerce Server vs MOSS 2007

2010-04-03T00:42:00.001+02:00

First of all, Mei Ling has written a great blog about getting Commerce Server 2007 working with MOSS, definately recommended reading.

There does seem to be some disconnect between the MOSS team and the CS team. This blog post looks into the areas of overlap of functionality.

Catalog Management
Commerce Server gives you rich catalog management functionality. Here, you could go down different paths. If all you needed was a catalog management function, you could easily define a few custom Content Types and some lists and you'd have the same functionality. However, you wouldn't have the integration with the basket and order management functions, so it’s probably best to leave it in the CS database. So here you have a choice, either:
a) Use the BDC to consume your CS catalog
b) Write your own CS web parts to use directly within MOSS
A combination of these last two methods are probably a good way to go.

Checkout Processing and Order management
I wonder why the CS team didn’t decide to rip out the (difficult to debug) pipeline framework and replace it with Windows Workflow. An alternative to the checkout processing and order management is to use InfoPath forms and workflow to put stuff in the correct databases, and send messages through BizTalk (or other service bus technology) to actually take the money and send orders to the fulfillment houses.

eMarketing
One of CS's other big functional areas is in eMarketing, and again we see more than one way of doing stuff. Do you use the CS targeting system or MOSS audiences? I’m not sure, but it’s at risk confusing my users with two different systems to target their users. I think audience targetting through MOSS is better, simply because we will want to target both products and content to the same audience groups.

Discounting System.
This is one of the stronger points of CS, and I can't see anything obvious in MOSS to replicate this functionality.

Reporting and Analytics
CS uses SQL 2005 Reporting and Analytics, so these reports can be consumed into a Report Center within an MOSS Intranet.

Membership and Role providers

The out of the box providers are hooked up to CS as well as in MOSS so this is no area of concern. Using Forms Authentication, you could reuse the same in a MOSS environment.

So in conclusion, use the catalogue management, the shopping basket and the discount system from CS whereas all the other CS functionalities could easily be handled by reasonable amounts of custom development work in MOSS.

© Copyright 2010, Tomas Elfving

Heimore Identity 2.0

2009-11-23T17:49:00.000+01:00

Preparing a Portal/IAM-presentation for the Heimore Identity 2.0-day on Wednesday, http://www.heimore.com/Nyheter/Arets-event-Identitet-20--Identity-Management-i-verksamheten-nu-och-i-framtiden/
© Copyright 2009, Tomas Elfving

Google has launched Go

2009-11-19T00:24:00.000+01:00

Google has launched Go, a new programming language focusing on concurrency, simplicity, and performance.

Go is open source. It uses an expressive C/C++-similar language with pointer but no pointer arithmetic. It is type safe and memory safe. However, one of its main goals is to offer the speed and safety of a static language but with the advantages offered by modern dynamic languages. Go also offers methods for any type, closures and run-time reflection. The syntax is pretty clean and it is garbage collected. It is intended to compete with C and C++ as a systems programming language.

It features mulitcore programming by providing lightweight concurrency allowing developers to create sets of lightweight communicating processes, called "goroutines". You can run many concurrent goroutines and you don't need to worry about stack overflows. Goroutines aren't threads, they are functions running in parallel with other goroutines in the same address space. It is very easy to launch parallel functions using the goroutines. This is one of the most interesting features offered by the language. It really simplifies concurrency for systems programming.

© Copyright 2009, Tomas Elfving

A Web service layer versioning strategy using deprecated methods

2009-11-18T22:30:00.001+01:00

Question: How to manage development, bugfixing and all type of both backward- and nonbackward compatible changes to a web service layer without suffering from escalating maintenance costs of having to manage lots of old versions?

Suggested solution: Do not allow different version of the WS interface. Never change namespace. Handle change by introducing replacement methods and mark the "old" method as deprecated. Leave the deprecated method in production for a grace period, giving all consumers sufficient time to move over to the new version. When all consumers have moved over, remove the deprecaded method altogether.

This strategy works for both WS and EJB interfaces.

See also http://java.sun.com/j2se/1.5.0/docs/guide/javadoc/deprecation/deprecation.html

© Copyright 2009, Tomas Elfving

Visio stencil with SOA-symbols

2009-09-21T07:37:00.003+02:00

As a compliment to his excellent book "SOA Design Pattrens", Thomas Erl also offers a Visio stencil with SOA symbols. Very useful!

Download from http://www.soapatterns.com/soa_08_12_beta.zip

MOSS Audience Targeting & Audience Rules

2009-09-20T09:27:00.007+02:00

To scope content in MOSS 2007 using audience targeting, You have three options:

1. Sharepoint Groups

SharePoint Groups are maybe the most obvious Target Audience mechanism. This is useful in situations where the site administrators may not have access to Active Directory, which generelly is the case in large organizations. SharePoint Groups have the nice feature of allowing self-enrollment. Self-enrollment is useful if the site administrator wants to setup a site that have different levels of information and allow the users themselves to subscribe to what components they'd like.

2. AD/Domain Groups

Active Directory domain groups are a valid Target Audience and it works also if you are using a Custom Authentication provider, You just use Custom Role provider as an audience. The advantage is that existing AD groups (e.g for internal use) can be reused in external sites. The SharePoint site administrator has less or no control over the membership in the group, but in large organizations, this is generally the way the security department wants it anyway :-)!.

3: Audience Rules

Audience Rules are very powerful and maybe the least understood. They can be setup to achieve a number of useful things. For instance , they can be setup with multiple rules to require a match to all rules or any rule. The rules can be as simple checking if the user belongs to a organization, is in a distribution list, in a security group or match a specific user profile property.

Consumer power on the Internet

2009-07-06T21:01:00.003+02:00

After filing support issues to IBM for ten months to not much avail, it took IBM only a few weeks after I blogged about it to get a version out addressing the problems with the Clearcase for Visual Studio 2008 plugin that we have reported (http://blog.tomaselfving.com/2009/05/clearcase-for-visual-studio-problems.html, comments). Coincidence? Great news anyhow, only that we have already moved to VisualSVN and are happy now, maybe we look at the new Clearcase version at some later point in time.

ROSS for multi-stage deployment and multi-farm replication for SharePoint

2009-06-14T09:12:00.005+02:00

When searching for an enterprise deployment solution for a large external MOSS platform, I've come across ROSS. RepliWeb Operational Synchronization for SharePoint (ROSS) claims to provide one thing that is missing in the MOSS box, namely support for multi-stage deployment and multi-farm replication.

SharePoint used as a mission-critical web platform is requiers IT to maintain rapidly growing infrastructures, content and application structures. RepliWeb’s ROSS addresses the challenges when managing content deployment and replication processes across multi-staged topologies and multiple farms. Agnostic to environment variations, ROSS is a scalable solution built for critical enterprise deployment processes including comprehensive scheduling, recovery and transactional deployment capabilities, efficient transfer engines, and much more.

The feature I am specifically interested in is the support for transactions, so that a deployment may be rolled back entiryly without messing up the environment.

I'll comment on ROSS again later when I have made a thorough evaluation.

Clearcase for Visual Studio problems

2009-05-02T08:27:00.004+02:00

I'm working with a large corp customer that have standardized on IBM Clearcase. It works great with all the Java development tools, but when starting up large scale Visual Studio development we have experienced lots of problems, especially with the Clearcase plugin for Visual Studio 2008. For instance:

- Clearcase looses files, especially when adding files to VS project locally

- Merge performs poorly in general. In particular, merge of the VS project file is a source of major headache. Merge of CSS files reports successful, but when we're analyzing it there was lots of errors in the merged file.

- The Update view command is unreliable. Developers doing the Update view command doesn't always get all the latest files, and worse is that they get no message. They work on old files without knowing it, the developers worst nightmare!

If you have any experiences or tips, feel free to contact me or comment!

MOSS sp 2 now available

2009-05-02T08:10:00.004+02:00

The MOSS sp 2 is now available for download at http://blogs.msdn.com/sharepoint/archive/2009/04/28/announcing-service-pack-2-for-office-sharepoint-server-2007-and-windows-sharepoint-services-3-0.aspx.

Major improvements in the areas of:

1. Performance, availability and stablilty - lots of small fixes and improvements across nearly all the components. New/improved functions like a timer job that automatically rebuilds content database index to improve database performance. Another difference is when a content database is marked as read-only, the user interface will be modified so users cannot perform tasks that require writing to the database.

2. Broader browser support. Internet Explorer 8 is added into Level 1 browser support. FireFox 3.0 is added into Level 2 browser support. (Firefox 2.0 is no longer supported by Mozilla)

3. Improved Forms based authentification. Now the client application can store user credentials instead of asking for them every time.

4. Long-awaited improvements in product dokumentation

All the details of SP2 can be downloaded here:

http://go.microsoft.com/fwlink/?LinkId=148551

SIEM and logging/tracking event by Heimore

2009-04-03T22:46:00.003+02:00

Focusing on SIEM (Security Information and Event Management), traceability, log management and regulatory requirements Heimore Group is on the 13th of May arranging a full day of presentations, case studies, and meetings round these topics. The leading software vendors are coming as well as a number of leading experts in these fields.

Sharepoint proprietary specs out

2009-03-23T23:18:00.003+01:00

Microsoft have released some interesting reading in the SharePoint Products and Technologies protocol documentation. It provides detailed technical specifications for Microsoft proprietary protocols (including extensions to industry-standard or other published protocols) that are implemented and used in SharePoint Products and Technologies to interoperate or communicate with Microsoft products.

The documentation includes a set of companion overview and reference documents that supplement the technical specifications with conceptual background, overviews of inter-protocol relationships and interactions, and technical reference information.

Enjoy by downloading it at http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=5e94ad07-902c-422f-aadd-ff2bba9e540a#tm!

Skype staus field in Sharepoint

2009-03-19T22:08:00.003+01:00

A company named ObjectConnect have developed a Skype integration into SharePoint.Its first version available on Codeplex now (http://skypestatusfield.codeplex.com/) featuring the Skype status field. It allows you to store the account name in the field, and it's rendered as the nice status (green / grey) picture in the list and item view.

Also, clicking on the icon will bring you the chat window to that person if you have Skype installed locally. Nice!

An architecture of base-system-independent web services

2009-03-10T00:14:00.003+01:00

One of the most common mistakes of SOA implementation is to leave out the common data model. Without that, the web service layer is no more than a function-mapping layer between clients and base systems. By letting the ESB manage a common data model, the web services published to clients are independent of various base system definitions of common objects, and also independent of change in theses base system definitions.

SharePoint 2007 Backup/Restore options and things to consider

2009-03-07T00:03:00.004+01:00

Going through the native Sharepoint backup, I've found some limitations I'd like to shed some light on. The backup:

does not Provide Scheduling Functionality. You must use the command line with Windows Task Scheduler for scheduled backups
does not Backup any Configurations/Customizations to any files in the “12 Hive” or Web.config files. You must manually back up front end files.
does not Backup IIS Settings/Configurations
does not Backup Alternate Access Mappings (AAM)
dannot Backup Directly to Tape (only to UNC)
high restore time means low availability

If using the SQL Only Backup/restore strategy, you'll have the following issues to consider:

it does not backup any WFE configurations or solutions
it requires Admin to manually reattach content databases to the web applications after a recovery
you'll need to manually backup/restore all customizations on WFE Servers (.Net Assemblies, Features, IIS Metabase, etc. – batch file can help automate this process)
no need to backup Search database (as it can’t be synchronized with Search Index)
backup/restore of SSP separately via SharePoint native backup/restore

First (?) Computer "Bug", 1945

2009-03-06T22:47:00.003+01:00

Version management of web services

2009-03-01T22:35:00.005+01:00

You basically have 2 choices: Either use separate URI's or use the "target namespace# for version numbering. Target namespace can also be used for multilanguage support.
Using the tagret namespace way together with a proxy or ESB, you can reduce client dependence on URI's. Say you have:

/Customer
/Customer/v1
/Customer/v2
/Customer/vX

/Customer always gives the client the latest version. The ESB acts here to sort out what WS is the latest version and supplies that WS to the client. Clients call /Customer for services they know is backwardcompatible.

/Customer/vX is for non-backwardcompatible services where the client needs to be sure exactly what version of the service he gets.

The "target namespace" solution follows the Amazon’s recommendation form publication of services (http://docs.amazonwebservices.com/AWSEcommerceService/2005-03-23/ApiReference/ServiceVersioningArticle.html)

© Copyright 2009, Tomas Elfving

What capabilities can a proxy in a SOA service layer handle?

2009-03-01T22:11:00.011+01:00

When implementing a SOA web service layer, a number of issues keeps reappearing in service after service. They should, in other words, be taken care of in a separate service proxy layer on top of the services. The proxy layer could be implemented as a custom developed layer or with an ESB (Enterprise Service Bus).

Contrary to the more classical enterprise application integration (EAI) approach of a monolithic stack in a hub and spoke architecture, the foundation of an enterprise service bus is built of base functions broken up into their constituent parts, with distributed deployment where needed, working in harmony as necessary. An ESB does not implement a service-oriented architecture (SOA) but provides the features with which one may be implemented. The ESB tries to remove the coupling between the service called and the transport medium.

The following capabilities a proxy/ESB can handle:

- Invocation - Support for synchronous and asynchronous transport protocols, service mapping (locating and binding)
- Ruoting - Addressability, static/deterministic routing, content-based routing, rules-based routing, policy-based routing
- Mediation and integration - Adapters, protocol transformation, service mapping
- Messaging - Message processing, message transformation and message enhancement
- Process Choreography - Implementation of complex business processes(think twice before getting into that as it ties you heavily to the ESB product)
- Service Orchestration - Coordination of multiple implementation services exposed as a single, aggregate service
- Quality of Service - Security (encryption and signing), reliable delivery, transaction management, SLA controls
- Management - Monitoring, audit, logging, metering, admin console, BAM
- Thread management - i.e limit number of concurrent calls to a base system

© Copyright 2009, Tomas Elfving

Sharepoint memory leaks, anyone?

2009-02-27T21:39:00.004+01:00

The SPDisposeCheck tool will help improve the quality of your SharePoint assemblies. It will inspect your SharePoint assemblies and check that you are correctly disposing of certain SharePoint objects (IDisposable objects which includes SPSite and SPWeb). The tool is based upon the guidance published in this MSDN article, Best Practices: Using Disposable Windows SharePoint Services Objects

This tool is not supported by Microsoft and is recommended to be used on Developer workstations and not on production SharePoint Server installations.

© Copyright 2009, Tomas Elfving

FAST Search for SharePoint available in beta 2nd half of 2009

2009-02-27T21:29:00.006+01:00

Microsoft have released a roadmap for the FAST search engine aquired by the norwegian search specialists some time ago. The new search server that will add the high-end search capabilities of FAST ESP into Microsoft Office SharePoint Portal Server, will be available as a part of the next release of MOSS.
For customers who are interested in the product, Microsoft also announced ESP for SharePoint, a special offering that allows customers to purchase high-end search capabilities today, with a defined licensing path to FAST Search for SharePoint when it becomes available.
The full name is "FAST Search for Internet Business". It will extend FAST ESP and provide a flexible platform for building engaging, search-driven Web site experiences. The product will be available in beta in the second half of this year and will feature new capabilities for content integration and interaction management, helping enable more complete and interactive search experiences.

© Copyright 2009, Tomas Elfving

What is a Claim?

2009-01-31T22:16:00.006+01:00

See also previous post on Claims-based authentification in next SharePoint

So, what is this claim and claims-based model that generates so much fuzz?

The claims-based model has three components:
1. the relying party, which needs the claim in order to decide what it is going to do
2. the identity provider, which provides the claim
3. the user, who decides what if any information he wants to provide.

Claims can contain static information such as birth date or credit card info, relationship-based information such as group membership or derived claims that make general assertions such as the user is over 21 years of age. There are also metaclaims about how information was verified, such as in-person registration, or how it was issued.

Claims can be used in three ways:
1. to securely transmit the requesting user’s identity across machines
2. provide application-specific concepts, such as roles, so applications can augment claims about the user and allow applications to reason about those claims in the context of authorization decisions
3. interoperate with multiple authentication providers in a consistent manner.

© Copyright 2009, Tomas Elfving

Claims-based authentification in next SharePoint

2009-01-31T22:15:00.008+01:00

Microsoft is replacing the authentication system for SharePoint Server and will instead rely on a new claims-based identity model.

The goal is to have SharePoint incorporate an authentication model that works with any corporate identity system, including Active Directory, LDAPv3-based directories, application-specific databases and new user-centric identity models(LiveID, OpenID) and InfoCard systems(Microsoft’s CardSpace and Novell’s Digital Me).

SharePoint will lose the current authentication system, criticised for its rigidity, in favor of using claims about a user, such as age or group membership, that are passed to obtain access to the SharePoint environment and systems accessed from Sharepoint applications.

The claims architecture, part of Microsoft’s Metasystem model for a distributed identity architecture, is based on such protocols as WS-Federation, WS-Trust and the Security Assertion Markup Language (SAML). The Metasystem includes an emerging technology Microsoft developed called Security Token Service (STS), which handles the exchange of claims.

Claims could be built dynamically, picking up information about users and validating existing claims via a trusted source. The advantage of a claims-based system is that it is flexible and designed for heterogeneous identity environments.

From a stratigic point of view, the interesting thing is that Microsoft hopes that the claims architecture, which can be built on technologies available today, will replace identity systems that are based on a single-point-of-truth, typically a directory of user information. Most of today's identity management system are based on the single-point-of-truth policy.

© Copyright 2009, Tomas Elfving

MOSS, now as WSRP Producer

2008-12-26T09:06:00.007+01:00

Until now, the WSRP support in MOSS 2007 have been limited to acting as WSRP 1.1 client by using the built in web parts. With the recently released WSRP Toolkit for SharePoint you can produce WSRP conformant data producers from SharePoint lists and libraries. External portal platforms (e.g. BEA AquaLogic Portal, IBM WebSphere Portal, SAP NetWeaver Enterprise Portal etc.) can then render SharePoint data natively through their WSRP consumer portlets. The Toolkit is available now for download from the MSDN Code Gallery.

© Copyright 2008, Tomas Elfving

Hot deploy in a MOSS web farm means loosing session state

2008-12-10T01:02:00.012+01:00

The problem
Manys blogs describe how to deploy new code to a MOSS farm, but I have so far not found any solution how to do this without affecting end-users' session state. The scenario is as follows:

Let's say you have 2 WFE's and a load balancer in front, the classic minimal MOSS-redundant arcitecture. You set the load balancer to direct all hits to WFE2 while deploying on WFE1. The user sessions on WFE1 that you are updating are thrown out. The sessions cannot be transfered to WFE2 as it is not running the same code.

Desired solution
What you really would like for your users is to continue running their sessions based on the "old" code on WFE1 until they are done. That could take hours until all remaining users are done.

The Weblogic solution
WebLogic has a feature for this called parallell deployment to eliminate downtime during EAR version upgrade. You can deploy the new version without stopping the existing application and once the new version deployed successfully you can switch over transparently from the old one to new one. MOSS have currently nothing similar to offer.
More info: http://edocs.bea.com/wls/docs100/deployment/redeploy.html#wp1022490

The severity of this problem obviously depends on the functionality of your site. For an e-commerce site the loss of user shopping baskets could mean significant business impact.

© Copyright 2008, Tomas Elfving

Using external IAM or the portal platforms OOTB security functions?

2008-11-01T01:03:00.004+01:00

I've done a few posts on the IAM/portal subject (http://blog.tomaselfving.com/2008/10/security-architectures-for-portal.html, for instance) and there is surely a lot to gain from separating the access management and the identity management functionality from the portal platform and its web applications. On the other hand, an IAM platform is a huge investment both to buy and to implement as it requires expert product competence.

So, what are the arguments for choosing an external IAM platform?

In my opinion, if You have requirements for...

1. Several login methods, especially login methods not found in portal platform products.

2. Protecting more than one appllication, especially applications on different platforms (i e web applications and SAP)

3. SSO between applications on different platforms, both web and other apps.

4. High security requirements can in itself motivate an IAM-platform, as you may move the portal platform inside the DMZ and only have the reverse proxy on the DMZ.

5. Protecting a SOA environment of web services as well as web applications may be an interesting scenario for an IAM solution

...then, You may go ahead and evaluate IAM products to compliment your portal platform.

And finally, look carefully at the quality of the adapters between the IdM products and your portal product. It will be in the interface between the IdM and the portal that You will find the toughest challenges of the integration work. If this interface is poorly constructed, the IAM implementation may end up limiting the functionality of the portal platform!

© Copyright 2008, Tomas Elfving

Implementing Sharepoint security

2008-10-18T02:53:00.006+02:00

There are two major approaches to implementing Sharepoint security:

1. AD groups
Create Active Directory groups for each site collection and optionally each subsite and object that will require unique permissions. Create a new SharePoint OU (Organizational Unit) within Active Directory. Put these new groups in this OU. Then add these groups to their corresponding SharePoint groups in SharePoint (Owner, Member and Visitor). Adding users to the AD groups may also be done through an externa identity management application.

Advantages: Control, as all changes to security happen in Active Directory. It is easy to add/update/delete people from groups. No individual user accounts are stored in SharePoint with regards to security.

Disadvantages: Extra setup work. Each site you create a new site collection or a subsite/object that requires unique permissions. You have to set up Active Directory first and then add the groups to SharePoint. If your SharePoint administrations and your Active Directory administrators are different people, they have to cooperate.

2. Sharepoint groups
Ignore Active Directory groups and let the SharePoint administrators and site collection administrators add people directly to the groups provided by SharePoint.

Advantages: Easier admin as it lets the SharePoint administrators quickly change security without needing to involve the Active Directory administrators. It's also easier to see the names of everyone that has the permissions for a SharePoint group right from the interface instead of having to go to the Active Directory administrators and having them look it up.

Disadvantages: Maintainability suffers. You can have individual user accounts all over your SharePoint farm and not know who has permission to what. Out of the box, SharePoint does not provide a tool that shows you all the permissions a specific user account has (although there are 3rd party tools that provide this). When someone leaves or you need to move their account between departments, for example, it can be time-consuming to update the permissions in SharePoint.

The first approach is clearly the better practice in my opinion.

External Access/Identity management
With companies setting up external Access management/Identity management platforms, you also need to look into how the Sharepoint security architecture maps into the corporate access/identity management architecture. Typically, You want to use the IdM application to tie users to groups (either AD or Sharepoint groups, both are possible). This way, both internal and delegated administration can be provided. Delegated administration can be offered for extranet partners to manage their own users for instance.

© Copyright 2008, Tomas Elfving

Security architectures for portal platforms revisited

2008-10-16T23:19:00.005+02:00

Do you want to secure more than one application? Manage identities efficiently? This post describes on a high level access and identity management solutions managing the security functions for a portal platform.

This pics shows an RSA implementation, but similar solutions can be set up using any of the leading AM/IdM products:

Identity management
The idea is to centralize the authentification, authorization, administration and delegation of user identities (to create, update, delete and block user accounts) and access control of to user data to a company-wide common platform of reuseable components, instead of doing that in each and every application. For instance, a user’s address may be changed and the identity management solution immediately propagates that into every application and system integrated by the identity management solution. In extranet solutions you may even delegate the user management of a partner company’s users to a superuser in the partner company’s organization by making the identity management solution accessible on the internet (this scenario requires the identity management solution to be used in conjunction with an access management solution).

Separating login methods
Login methods (such as userid/pwd, digital certificates, smartcards etc) may be added to a separate login service. The access management platform ties different login methods to different applications. By choosing this way, you don’t use the built-in security functions of the portal product at all. You trust the access management platform to secure all portal applications. The access management platform may also be used to secure a web services environment in a SOA architecture implementation. Yet another scenario is to secure both an existing web site and a new site during migration.

Web portal application integration
One problem is that editors and developers of web portal applications needs to work in their portal platform (MOSS 2007, for instance) when creating groups and giving access to various resourses of their web application. It might also be complex with rolebased access rights and other fine grained access solutions. You need some interface definition and integration solution between the Identity management solution and the portal platform.

I suggest having the portal platform responsible for creating groups and access rules for resources. The IdM then extracts the groups. In the IdM You add people to the groups which will give them correct access to resources when logging into the web portal application via the Access Management component (which integrates to the IdM as well).

© Copyright 2008, Tomas Elfving

Thoughts on SOA Governance

2008-09-24T23:58:00.002+02:00

Some advice when it comes to the creation process of (web) services for a SOA platform:

1. Only create services from core business logic/funktioality. Where is an overhead in terms of work requiered creating and mainatining web services so focus on the ones most important for your business

2. Bigger is better. Only the really big services qualify as SOA services for reuse througout a company.

3. Don't make a services per function. Again, bigger services with mony functions is easier to reuse and maintain.

4. Document each service separately from other development documentation. That makes the documentation of the service more accessible to its users and to people whating to make changes later on.

5. Create unique SLA's for each web service. The reason is that each web service may be used in different applications with different requirements.

When it comes to management of SOA I believe it's a good idea to separate creation from maintenance of services. I suggest you separate SOA governance in two parts:

1. Create services - The initial decision when you choose what will be a SOA web service and what doesn't is taken by the company's IT architecture board.

2. Change management of services - When a service is changed it may affect others, therefore a Change Control Board for SOA is necessary. That board controls any changes in webservices and they prioritize the development work.

© Copyright 2008, Tomas Elfving

SSO between a MOSS and a Weblogic web site

2008-09-01T23:52:00.007+02:00

The problem

I need to set up single-sign-on between two web sites. The new site is in MOSS 2007. I am migrating to the MOSS-based site from the existing old site that is built in BEA Weblogic with a SUN One web server in front. The MOSS site requires login. The existing site contains applications requiering login (using the Weblogic realm). It is too expensive to migrate the entire site to MOSS at once, so we need to do a migration in several steps having both the old and the new site in production during the overlapping time.

Proposed solution

I am considering IFrameing the existing applications into the new MOSS-based site while migrating them one by one. The problem is that I cannot expect the user first to log on på the MOSS-site and then to logon to the IFramed applications residing on the existing Weblogic site, so we want SSO between them. I am considering using ISA Server (or maybe another Access management product) to achieve this. With ISA Server the both servers need to be in the same domain and use the same Web listener. I think the solution looks good on paper, but the devil is, as always, in the details. I'll write more on this when I have tried this out in reality. If You have any experience on something similar, i'd appreciate any comments on this!

© Copyright 2008, Tomas Elfving

Implementing Pluggable Single Sign-On in MOSS 2007

2008-07-18T01:43:00.006+02:00

The Microsoft Single Sign On Service is one of the most misunderstood and overlooked features of SharePoint. Many people think of it as a federated SSO mechanism by which one would gain a "one sign on for several applications" environment whereby all third party apps could be assimilated into the SharePoint environment using an arbitrary sign on mechanism. It's purpose is rather to integrate and connect to third party base systems, such as SAP, Siebel, or other LOB systems.

The power that it provides, when combined with the Business Data Catalog (BDC), is to eliminate the need to build complex identity impersonation in order to retrieve business data out of those third party systems. Using its structure of Enterprise Application Definitions (EAD) one can build several pass through authentication channels by which business data can easily be assimilated and brought to a web front end application through the use of various webparts, whether it is OOB, or a custom developed solution.

The Default Single Sign On Service
Single sign on in SharePoint needs to be called up by using services.msc. Once you get started, the best next step is to start to plan out how you are going to configure the accounts for SSO, which are:

Configuration account
Single sign-on administrator
Single sign-on service
Enterprise application manager

You can set the exact criteria of what needs to be satisfied for each account. After the accounts are created you will create the encrypted SSO database where we are going to map and store our third party credentials. Once your database is created, and the encryption key is created, you can create your EAD which is going to define the structure. It says exactly how the retrieval of the credentials are going to be handled.

Pluggable SSO Architecture
The Microsoft SSO Service API is independent of the specific authentication mechanisms. The unique feature of the new MOSS SSO is that the Microsoft Single Sign On service is now a pluggable feature, by which you can leverage whichever SSO provider that you would like. It may be a custom one that you have developed or one provided by a third party. When deploying your own or a third party provider, the default SSO provider, SpsSsoProvider, will no longer be used. Because only one SSO service can be used, it is a one or other choice. A MOSS administrator/architect has to decide whether they want to bind the default SSO provider or use one of their own choosing, each which carries its own implications.

© Copyright 2008, Tomas Elfving

Next Generation Portal – Federated Portals

2008-07-08T00:07:00.004+02:00

In order to provide the user a single business platform for all enterprise resources, the user should be able access content across all portals in the enterprise. To achieve this, portals have to be integrated in such a way that content is location-transparent to users without users having to remember numerous URLS and identities for access. This is the next generation evolution for portal computing. Gartner refers to such portals as federated portal. Such federated portals need to provide federated content and identity (single-sign-on).

This posting discusses industy efforts and standardizations that drives this development.

Content (Application) Federation

JSR 168

JSR 168 is outcome of Java community process, an open forum for the development of Java standards, reference implementations and TCKs (Test Compatibility Kit). JSR 168 is very similar to Servlet specifications, the request-response presentation framework part of J2EE, the java enterprise edition middleware standard. JSR community made a decision to create a new specification for portlets. JSR 168 defines a standard for creating application markups (aggregate-able application interfaces) called portlets. It also defines a run-time environment service for portlets called portlet container and a protocol API (application programming interface) for portlet and portlet container handshake protocol.

Portal container provides lifecycle management for portlets, processes the user requests and generates dynamic content to form the user desktop. The portlet container to portlets is analogous to servlet container to servlets. Portal container leverages servlet technology and can be built on top of the servlet container. Portlet container differs from servlet container by adding presentation constructs with portlets. Presentation constructs enable the container to produce an aggregated user desktop and allows the container to send information to portlets on user actions. Presentation constructs include predefined portlet modes and window states. Since the portlets need to be personalized, container services include a declarative way of providing user information or attributes to the portlets.

JSR 168 provides declarative security model for the portlets but does not favor any single sign-on standards. This specification follows J2EE role based declarative security model for portlets. It also includes standard deployment constructs and conformance toolkit for portlet container services.

This standard achieves several milestones in portal computing. It provides:

a) A standard API for creating portlets or dynamic application content markup for portals
b) A standard container service model – Container services can be provided my as off-the-shelf system vendors (portal servers), allowing developers to work business applications
c) No vendor lock-in – portlets could be run in any containers that conforms to this standard
d) Hot deployment of new portlets and hence no down-time for new application launch
e) Support for localization and internationalization and
f) Support for multiple types of client

WSRP – Web Services for Remote Portlets

WSRP is the outcome of joint efforts of two Oasis (www.oasis-open.org, a non-profit ebusiness standards organization) technical committees, WSIA – Webservices for Interactive Applications and WSRP itself. WSRP 2.0 has been approved as standard in April 2008.

WSRP addresses the issue of location transparency for portlets. WSRP leverages webservices infrastructure for accessing the portlets hosted in remote containers. Location transparency is not new to the computing world. Several technologies were invented to do this, starting with RPC remote procedure calls. During component Object oriented paradigm revolution, DCOM, RMI, CORBA and EJB were invented to address this. Webservices is becoming the latest in this line with its advantages to work over the Internet.

WSRP defines a producer-consumer-client architecture for getting applications to user desktop. Client refers to the browser that acts as a user desktop. Consumer refers a portal server or aggregation engine that aggregates from several producers to form the user desktop. Producer can be application interfaces (other portals serving portlets or stand-alone applications) that produce markups. WSRP defines producer constructs for service producers; consumption constructs for consumers and a protocol for consumer-producer communication.

How WSRP works

Portlet state in the consumer is part of transport and hence consumer doesn’t have to maintain this state but producers can be mandated to consumer to handle session persistence. Consumer communicates to producer via SOAP (Simple object access protocol). Producer publishes the interfaces services or applications via WSDL (Webservices definition language). Consumer can lookup, register with the producers and invoke their services. Consumer maintains session with client.

WSRP defines a Portlet management interface allowing the consumers to customize the portlets and refers them as consumer configured portlets. This allows producers to cater different set of consumers by allowing themselves to be customized by the consumer consuming it. WSRP also address URL rewriting since consumer aggregates content from producers (portlets) that can be remote behind a firewall. An example of this could be third party services provider acting as WSRP producer. In this case, remote portlets may have produced URLs that point back to itself and since they are remote, the consumer (portal) would need to rewrite the client URLS. WSRP delegates the security implementation between the consumer and producer to webservices security standards

Identity (Single Sign-On) Federation

JSR168 provides standardized local portlet integration to portal desktops. WSRP allows them to be location transparent enabling applications to be shared across portals. Next issue to address is federation of declarative security systems built for portals called as Web-SSO (Web Single Sign-On).

Web-SSO model has to evolve into Federated SSO model solutions allowing user to navigate between different portal domains without having to re-authenticate. SAML exactly addresses this topic of access security inter-operability between different security systems, essentially portals.

SAML – Security Assertion Markup Language

SAML, currently in its version 2.0, specification from Oasis technical committee is an XML framework for exchanging assertions between security systems. Assertions are verification of certain facts about subjects; subject being a user or a system.

The specification has four major components:
a) assertion and request-response protocol data formats called SAML Core
b) bindings and profiles called SAML Bind
c) conformance program for SAML called SAMLConform
d) security and privacy considerations called SAMLSecure

SAML Core covers three types of assertions:
a) Authentication – Was the subject authenticated by a particular means at a particular time?
b) Attributes – Is the subject associated with supplied attributes?
c) Authorization – Can the subject access a specific resource?

SAML Core defines a request-response protocol called SAML protocol between assertion artifact (token) issuing party and the relying party. Relying party asserts with Issuing party about user log-ins, attributes and authorization credentials. Specification is designed in such a way that this request-response protocol can be used in several scenarios and leverage transport protocols for communication. In SAML terms, scenarios are referred to as SAML Profiles and transport protocols for communication are referred to as SAML Bindings.

Perspectives

JSR 168 will allow application developers to write portlets allowing integration with any Java-based portal server providing container services. It will also create a market for portlets. Microsoft Web parts is a competing, proprietary but successful, technology on the .NET platform.

WSRP adoption is emerging and has the potential to be the standard that bridges the Java-.NET gap.

WSRP and JSR 168 is set to enable Content federation.

Adoption of SAML by identity market is evident as vendors have rush to implement this. IBM Tivoli, Oblix, Sun, SiteMinder, Securant, Evidian, Entrust, Netegrity, RSA Security, Baltimore technologies, Sigaba, Verisign are some to name a few. SAML browser profile compliant Access Security System is set to address Security Federation
Examining the relations between these standards, JSR 168 does not mandate WSRP but refers it to remote portlets. WSRP does not mandate JSR 168 portlets but JSR 168 fit right as portlet standard in the java world. Since WSRP does not mandate JSR 168, application (markups/portlets) can be aggregated between both .NET and Java. WSRP not directly need or mandate SAML, but the need for SAML arises in the case of federated single sign-on. SAML does mandate SOAP over HTTP in its implementation.

The missing link - Federated SSO for webservices

Taking a closer look, missing standard is federated SSO for webservices. WSRP rely on webservices standards and hence federated SSO would be necessary to consume a WSRP producer produced portlet, protected by a different Web-SSO system. Example of this could be someWSRPConsumer.com portal accessing a portlet on someWSRPproducer.com and both the portals have their own Web-SSO systems.

SAML was designed smart to allow applications with different scenarios. Webservices security model standards could use SAML for achieving this federation. WS-Security, security language of webservices created by IBM and Microsoft defines a SAML profile for SOAP messages. WS-Security is not discussed in detail in this posting. WS-Security is yet to become an open standard but under consideration. WS-Security SAML profile can provide federated-SSO for webservices.

Conclusion

Portal has evolved as a user-centric application integration platform. It has become a strategy rather than solution implementation.

Federated Portals is poised to become the next generation concept for portal computing. Content and Identity federation will become the key technology enablers for federated portals. Without the standards, this will become a pipe dream. JSR 168/Web parts, WSRP and SAML are set to achieve these goals. Co-existence with federated content and identity standards will become one of the key selection criteria for all off-the-shelf web solutions.

In order to reach to the next generation, businesses would have to conduct assessments to identify federate-able portals. The value of consolidating infrastructures for application and content delivery has proven in case of portals. Federation follows its footsteps at the enterprise level avoiding multiple user identities even between portals, eliminating the need for same application launches on different portals. In some cases, such federation model can even become business opportunities. An example of this could be an enterprise participating in the federated ebusiness eco-system acting as a service provider.

Planning, Choosing, Aligning and Governing were the key elements of a successful portal implementation. In case of federated portals, challenges are no longer at the departmental levels, but at the higher level, enterprise-wide.

© Copyright 2008, Tomas Elfving

Hotfix for SharePoint group target audience issue available

2008-06-25T22:54:00.004+02:00

As one reader pointed out (thanks!), there is a Microsoft Office SharePoint Server 2007 post-Service Pack 1 hotfix out now adressing an issue that I wrote about in a post last year. The symptom appears when You add Active Directory user accounts and Active Directory group accounts to a group in MOSS 2007. When you designate the SharePoint group as the target audience for a specific Web Part, some user accounts in the SharePoint group cannot access the Web Part.

Check out the hotfix!

© Copyright 2008, Tomas Elfving

Poor integration between Oracle Webcenters portal and content management tools

2008-06-25T22:52:00.002+02:00

The built-in CMS in Oracle Webcenter (UCM), formerly Stellent, is very powerful for building and maintaining ”information-heavy" sites with its great document management and workflow features. The problem is that the UCM has no portal and portlet integration functionality.

To edit the site structure of a web portal application (change meny items or order, add/remove portal web pages and change portlets on web pages) You have to use the development environment (typically JDeveloper). Then You have to rebuild and redeploy the web application onto the target production environment. This means that web site editors are restricted to edit content in an existing site structure, otherwise they have to engage developers to build and deploy. A very time- and resourceconsuming process. These type of changes simply shouldn't require a re-deploy in my opinion. Competing portal products have better solutions. Also, it is obviously a disadvantage to have to work in two separate tools with content updates.

I have heard mixed messages from Oracle regarding this, I have both got information saying that they will fix this, but I have also heard argumentation from Oracle experts that this "flexibility" is an advantage!

© Copyright 2008, Tomas Elfving

WSRP 2.0 support in MOSS and Oracle WebCenter

2008-05-29T06:02:00.005+02:00

The long-awaited (5-6 years since WSRP 1.1...) approval of the WSRP 2.0 specs have arrived. Microsoft have promised support for it in MOSS 2007, probably by adding an OOTB Web part working as WSRP 2.0 Consumer (the product has a WSRP 1.1 consumer today). No time plan for that work have been communicated though. I have also evaluated Oracle's Webcenter recently and it turns out WebCenter uses WSRP 2.0 for all portlet-to-portal container communication, even in cases where the portlets resides on the same server as the portal container!

. © Copyright 2008, Tomas Elfving

Using the WSRP 1.1 Consumer Web part in MOSS 2007

2008-05-29T05:43:00.006+02:00

Having Java/JSP apps too expensive/timecomsuming to migrate to MOSS 2007? Maybe the apps works just fine as is, and the migration adds no business value. Besides IFraming them, the way to go is WSRP. Check out this architecture picture:

Package your existing apps as WSRP producers and use the MOSS OOTB Web part acting act a WSRP 1.1 consumer. Some advice on the way:

1. Keep portlets simple - In general for MOSS and WSRP, if you keep the portlets fairly straightforward, you should not run into problems. If you start to use a lot of script, DOM modification, AJAX calls, etc. then you may run into issues.

2. Start small - I would try a “real” part of the application that includes navigation, form submission, script, user identity, session state and persistent state.

3. Namespacing - Microsoft WSRP implementation is not great. MOSS uses an IFrame to implement it’s WSRP browser side container so you typically do not have to be concerned with namespacing. In fact, you may run into problems if you try and use “wsrp_rewrite” prefix (consumer namespace re-writing). There is a bug in their namespacing. You may need to rely on producer namespacing instead (if you plan on consuming those portlets elsewhere or back in your Java Portal server).

4. Test - Thoroughly test WSRP functionality in MOSS first to uncover issues first.

© Copyright 2008, Tomas Elfving

Java/.NET web service interoperability issue regarding checked exceptions

2008-05-07T23:32:00.004+02:00

Using web services as integration technology is an obvious choice when service-orienting your companys IT architecture, especially when having a mixed Java and .NET environment. One strategy many companies choose is to package business logic in Java-based web services while developing web user interfaces on the .NET platform.

The problem

There is one interoperability issue that I'll like to shed some light on, and that is "checked exceptions". Checked exceptions is userdefined exceptions. Java supports this construct whereas C# doesn't. The C# development team have chosen to exclude checked exceptions by design (why? -> pls read the interview with the lead C# architect Anders Hejlsberg at http://www.artima.com/intv/handcuffs.html). This imposes a risk of having unhandled exceptions. A java-based web service may throw checked exceptions that the .NET-based client is unaware of. The .NET client developer needs to manually analysis the WSDL file to see what type of exceptions the web service method may throw. This is a tedious and error-prone way to work compared to the Java/Axis that autogenerates a complete interface including exceptions in object-form on the client side.

The solution

C# doesn't support checked exceptions by design, but Windows Communication Foundation supports automatic analysis of the WSDL-file when creating the proxy on the client (you must use svcutil.exe, not wsdl.exe. Svcutil.exe is default in Visual Studio 2008, but not in 2005. Or run on the command line). Exceptions are serialized in objects that can be analyzed and handled on the client without needing to parse XML-strings for error codes and messages. The exceptions are catched in regular try-catch blocks just as any other .NET Exception. This way, you get type safe userdefined exceptions. You'll also get the "intellisense" in Visual Studio on exception objects.

© Copyright 2008, Tomas Elfving

Security architectures for portal platforms

2008-04-30T22:58:00.004+02:00

[16 oct 2008 - There is a new better version of this post at http://blog.tomaselfving.com/2008/10/security-architectures-for-portal.html.

Regards, Tomas]

Do you want to secure more than one application? Manage identities efficiently? This is a first post in a upcoming series around the subject of security in terms of access and identity management solutions managing the security functions for a portal platform.

The idea is to centralize the authentification, authorization, administration and delegation of user identities (to create, update, delete and block user accounts) and access control of to user data to a company-wide common platform of reuseable components, instead of doing that in each and every application. For instance, a user’s address may be changed and the identity management solution immediately propagates that into every application and system integrated by the identity management solution. In extranet solutions you may even delegate the user management of a partner company’s users to a superuser in the partner company’s organization by making the identity management solution accessible on the internet (this scenario requires the identity management solution to be used in conjunction with an access management solution).

Login methods (such as userid/pwd, digital certificates, smartcards etc) may be added to a separate login service. The access management platform ties different login methods to different applications. By choosing this way, you don’t use the built-in security functions of the portal product at all. You trust the access management platform to secure all portal applications. The access management platform may also be used to secure a web services environment in a SOA architecture implementation. Yet another scenario is to secure both an existing web site and a new site during migration.

Setting multivalue properties in User profile

2008-04-07T23:10:00.006+02:00

More on User profiles; MOSS 2007 supports multivalue properties in the User profiles, but how do you set these fields? The this[] operator on the UserProfile object returns an ArrayList called UserProfileValueCollection. This code example shows you how to add multiple values to a multivalue property:

Add references to Microsoft.Office.Server, Microsoft.SharePoint and System.Web in your Microsoft Visual Studio project.

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.Office.Server;
using Microsoft.Office.Server.Administration;
using Microsoft.Office.Server.UserProfiles;
using Microsoft.SharePoint;
using System.Web;

namespace UserProfilesDemoApp
{
class UserProfileProgram
{
static void Main(string[] args)
{
using (SPSite site = new SPSite("http://servername"))
{
ServerContext serverContext = ServerContext.GetContext(site);
UserProfileManager userProfileManager = new UserProfileManager(context);
UserProfile userProfile = userProfileManager.GetUserProfile("domainname\\username");

// Insert your database access code here to get the values to set!

userProfile["MyInterests"].Add((Object)"Soccer");
userProfile["MyInterests"].Add((Object)"Icehockey");
userProfile.Commit(); }
}
}
}
}

© Copyright 2008, Tomas Elfving

Master and supplementary data sources in Sharepoint 2007

2008-03-18T23:47:00.007+01:00

The user profiles are a neverending source of issues, it seems! Get e-mails with all sort of questions regarding populating and using the user profile. Today, I’ll address the concept of master/supplementary data sources.

The basics
The User profile store contains the user account property information. This information is obtained by importing it from a directory or a data source that contains user accounts. MOSS 2007 can import a list of domain users from the Active Directory directory service, LDAP server, or the Business Data Catalog. In addition, you can write code against the object model to import information from other directory services or applications. You can schedule regular imports to the user profile store, and these can be incremental or full.

Master and supplementary data sources
But what if you need more data fields in your user profiles than your Active directory contains? What if you for instance have additional data in database or retrieved by a web service that you want to use for personalization or audience targeting? Office SharePoint Server 2007 treats Active Directory and LDAP directories as master connections for importing user information; that is, it can use them as a source to create user profiles. This master connection is responsible for adding and removing users from the user profiles database.
To add supplementary user profile info you use one or several supplementary data sources. They cannot add or remove users (like the master connection), but they can provide additional user information not available in the master connection. A Business Data Catalog data source is a typical supplementary data source.

There is a "preliminary" sample under way at http://msdn2.microsoft.com/en-us/library/ms585895.aspx. Also check out http://blah.winsmarts.com/2007-4-SharePoint_2007__BDC_-_User_Profiles.aspx.

© Copyright 2008, Tomas Elfving

Portal platform with integrated ”e-legitimation” login for processoriented organizations and businesses

2008-03-06T00:52:00.005+01:00

To celebrate the re-launch of Heimore Groups site at www.heimore.com, I’d like to take the opportunity to give You a brief about one of our “software-as-a-services” - ExpressPortal.

Note: e-legitimation is the Swedish digital ID solution, commonly accepted by governmental organizations, banks etc.

ExpressPortal - A Portal platform with integrated ”e-legitimation” login for processoriented organizations and businesses

ExpressPortal is a ready-to-run portal framework solution for both intranet and external users. The solution features all the out-of-the-box functionality that you’d expect in a modern portal platform (enterprise search, content management, personalization, role- and target group based interaction for instance). The Heimore Group ExpressPortal solution is tested and ready-to-run with built in components for digital signatures, e-legimitation access control, authorization and SSO (other access control systems can also be plugged-in).

ExpressPortal builds upon an open, scalable and extensible architecture based on market-leading portal products dedicated to openness and portal standards such as Web parts, JSR 168/268 and WSRP, which makes the portal platform easy to integrate to and from. It has an open architecture making it ready to use to implement portals and applications building on services in a SOA-based environment to support processoriented businesses and organizations.
The solution contains a framework to efficiently assemble portals and applications. It features out-of-the-box templates for common eServices and applications, preinstalled integrations with digital signatures and e-legitimation, content management and typical business processes.

Curious? E-mail me for more info, or check out the (only swedish at the moment, sorry for that, translation is in the making) ExpressPortal whitepaper, and our other packaged services named ExpressInfrastructure.

Figure 1: The ExpressInfrastructure Architecture

© Copyright 2008, Tomas Elfving

Hosting the Windows Live Messenger IM Control

2008-02-26T23:03:00.005+01:00

It is very easy to integrate the Microsoft Live Messenger Control like I have done on this blog. Just follow the steps described in the MSDN article at http://msdn2.microsoft.com/en-us/library/bb936683.aspx.

© Copyright 2008, Tomas Elfving

Office SharePoint Server 2007 Capacity Model now shipping

2008-02-16T06:55:00.017+01:00

System Center Capacity Planner 2007 has shipped, now with two New SharePoint deployment models!

Office SharePoint Server 2007 Capacity Model
Windows SharePoint Services 3.0 Capacity Model

By supplying information such as...

number o users
internet vs. intranet,
distribution between anonymous vs. authenticated users
usage profiles (Light <-> Heavy publishing, Light <-> Heavy collaboration)

...you get planning help and high level guidance regarding topology and hardware requirements. You can analyze how your deployment may be impacted by a merger or other scenarios and variabels.

This tool simplifies the hardware acquisition and address some of the early number of server licensing type questions. Note artificial limits have been placed in the tool to limit it at 100,000 users and 3 TB. Above these limits, Microsoft recommend MCS or experienced Architects be consulted. Even up to these limits it's recommended to consult with experienced architects and consultants to validate the recommendations from the tool. Also, check out the HP capacity planning tool to validate these results.

I took the tool for a spin. For a single-site minimum topology for a light publishing/light collaboration internet site with 50% anonymous and 50% authenticated users, the following spec is recommended:

Topology
Sites with servers: 1
Sites with clients only: 0
Total number of clients: 1

Site: Mini
Number of users: 1
Number of servers: 4
Number of SAN connections: 0

Server: Index Server
Processor: 2-processor, 2,40 GHz, Opteron (2-chip x 1-core)
Minimum memory: 8,0 GB
Disk: DiskArray 1\Volume 1 (File System), 144 GB RAID 5 (3 x 72,00 GB SCSI 10 000 RPM)
NIC: 1 x 1 000 Mb/s
Roles: Index

Server: SQL Server
Processor: 2-processor, 2,40 GHz, Opteron (2-chip x 1-core)
Minimum memory: 16,0 GB
Disk: DiskArray 1\Volume 1 (Log Files), 146 GB RAID 1 (2 x 146,00 GB SCSI 15 000 RPM)
DiskArray 1\Volume 2 (Data Files), 584 GB RAID 10 (8 x 146,00 GB SCSI 15 000 RPM)
NIC: 1 x 1 000 Mb/s
Roles: SQL Server

Server: Web Front End 1
Processor: 2-processor, 2,40 GHz, Opteron (2-chip x 1-core)
Minimum memory: 8,0 GB
Disk: DiskArray 1\Volume 1 (File System), 146 GB RAID 1 (2 x 146,00 GB SCSI 10 000 RPM)
DiskArray 1\Volume 2 (File System), 144 GB RAID 5 (3 x 72,00 GB SCSI 10 000 RPM)
NIC: 1 x 1 000 Mb/s
Roles: Web Front End; Query Server

Server: Web Front End 2
Processor: 2-processor, 2,40 GHz, Opteron (2-chip x 1-core)
Minimum memory: 8,0 GB
Disk: DiskArray 1\Volume 1 (File System), 146 GB RAID 1 (2 x 146,00 GB SCSI 10 000 RPM)
DiskArray 1\Volume 2 (File System), 144 GB RAID 5 (3 x 72,00 GB SCSI 10 000 RPM)
NIC: 1 x 1 000 Mb/s
Roles: Web Front End; Query Server

Personally, I think this seems a bit much... I mean isn't 4 double-processor servers as a minimum startup configuration a bit steep? It obviously depends on your availability requirements, but if You can skip the redundancy, You should be fine with 2 servers (one web and one database).

© Copyright 2008, Tomas Elfving

Massive support for OpenID

2008-02-09T07:44:00.000+01:00

I've over the last months done a number of posts on the emerging de-facto standard for internet authentification OpenID. Now the OpenID Foundation is announcing that Google, IBM, Microsoft, VeriSign and Yahoo! have taken seats as the organization's first corporate board members.

OpenID is appealing as it allows users to authenticate their identity through a single chosen provider instead of creating unique accounts at every website you use. Today, more than 10 000 web sites and 350 million users use OpenID.

For users, OpenID means much easier account creation, better personalization, privacy and security when trying out new web sites. It makes for a greatly improved user experience. For websites and other companies, OpenID means more and happier users and potentially greater access to information about those users.

There's a whole lot of momentum right now for OpenID. As I've previously told, in January Yahoo! increased the number of OpenID enabled user accounts by orders of magnitude, the long-awaited OpenID 2.0 spec was just recently finalized and the entire Data Portability paradigm is moving into the public consciousness quickly.

All of that said, big vendors have a lot of short term interest in controlling identity silos. It won't be easy to get their long term interests in openness to prevail. Fortunately, they are participating but are in the minority on the OpenID Foundation board.

There are many, many places you can get an OpenID and there are significant differences in advanced feature sets. To get a good look at the range of options and details beyond mere simple one-way authentication, check out the vendor comparison at SpreadOpenID.org.

Security features in MOSS 2007

2008-01-30T05:26:00.001+01:00

MOSS will often be implemented in an environment with an existing corporate-wide access management solution handling identity management, authentification and authorization. They typically work on a URL level for web applications. MOSS web applications will need to make good use and cooperate well with IAM/IDM solutions. This article looks at security features available in MOSS 2007 explaining the tools available on the portal level to leverage these technologies.

1. Pluggable Authentication Provider
An authentication provider is a component that lets you verify user credentials. The authentication provider is an important security component when setting up Internet-style SharePoint® authentication. MOSS supports the Windows®-based authentication methods available in previous SharePoint versions such as Integrated and Basic authentication, forms-based authentification as well as plain-text userid/pwd. You can add multiple authentication providers so that, for example, internal users log in via Windows authentication while external users do so with a separate, pluggable provider.
MOSS is built upon ASP.NET 2.0, which enables the use of pluggable authentication providers. This lets you use configurable directories for storing user information as long as there's an ASP.NET 2.0 membership provider (and optional role provider) that corresponds to the member data store. Pluggable provider credentials can be hashed, encrypted, or stored in plain-text depending on the node values stored in the machine.config file that correspond to the membership provider. Several membership providers are available for use with MOSS, some of which include an LDAP V3 membership provider, plus a SQL Server™ membership provider and Active Directory® membership provider (available with ASP.NET 2.0). Using the ASP.NET 2.0 membership architecture, it is feasible to create custom providers that use membership stores like Oracle databases, XML files, or even flat text files. A custom authentication provider inherits from the ASP.NET MembershipProvider interface, which in turn inherits from the ProviderBase class (see Figure 1).

Figure 1: .NET Membership Class hierachy

Note that those services related to crawling and indexing content, must use a Windows account even when using a pluggable provider.

2. Forms-Based Authentication and Web Single Sign-On
Forms-based MOSS authentication is based on ASP.NET 2.0. It utilizes pluggable authentication and role providers to enable Internet-style security mechanisms that don't restrict customers to an old-fashioned NT login prompt. Instead, you can build a customized login process geared toward your users' needs. To protect the authentication process further, forms authentication cookies and authentication tickets are encrypted and tamper-proof.

The form identity provider does not have to reside locally and can, in fact, plug into an external identity management system, which in turn would provide the validation of the user credentials. This functionality is known as Web single sign-on, and it employs an HTTP module for external authentication. In such a scenario, you can federate identities between your organization and business partners whereby they are able to use their own organizational logins to authenticate to your MOSS implementation.

3. Encrypting MOSS Application Connection Strings
It's not good a security practice to place valuable connection string information in plain text in a web.config file. The exposed information could enable a user to create an object instance to the SQL database server and manipulate valuable data. However, using inherent ASP.NET 2.0 functionality you can encrypt the data using either of two available technologies: Windows Data Protection API (DPAPI) or RSA encryption. DPAPI capitalizes on the built-in cryptographic functionality of Windows to encrypt and decrypt using the MOSS server machine key. RSA encryption lets you use public key algorithms, but carries the added overhead of needing appropriate containers for the encryption keys.

Using encryption does have some consequences. Because you are using the local machine key for encryption, you can only use this configuration node on the MOSS server it was created on-otherwise the cipher process will fail. If an intruder gained access to the server, they could retrieve the machine key and decrypt the connection string. The decryption process will also cause a minor application performance hit.

4. Alternate Access Mapping and Zones
Alternate Access Mapping (AAM) is useful when setting up more than one entry location for your MOSS environment. AAM simply enables you to specify different URLs through which users can access a single physical site (see the diagram in Figure 2). AAM can direct users to the same physical path for your MOSS site, but with different authentication providers and Web application security policies. AAM also compensates for using different domains, reverse proxies, and other URL redirection mechanisms. The URL that MOSS will use is already mapped for you by default, but this can be extended to additional URLs that a SharePoint architect explicitly creates to handle different methods of entry. AAM is required to ensure that proper internal and public URL mapping works correctly.

Figure 2 URL Mapping and Zones

In MOSS, zones are a new feature that allow greater control over the site mapping provided by AAM. Zones let you map different authentication providers to the same physical path and MOSS content depending on the AAM URL mappings through which users are connecting to the content. When specifying AAM URL mappings, although it is not necessary that the zone is bound to an authentication mechanism, it is recommended. An AAM URL mapping to a zone that is not defined in the authentication providers page will use the security setting for the Default zone.

Zones do require some planning since they will heavily impact how people are authenticated and routed through your portal from multiple URL entry points. When extending a new Web application, you can specify the zone you want to use under the Load Balancing URL section of the settings (see Figure 3). It is recommended that you place your most publicly accessible URL (for example, the URL you plan to expose to the Internet) in the Default zone, as this is the URL SharePoint will use if it cannot fully determine which zone you are in.

Within each zone you can bind a global Web application security policy that defines permissions settings for users in the zone. This lets you centrally manage large-scale permissions modifications. Web applications commonly include several sites in various site collections, and permissions management of these sites can become problematic when applications become large and complex. MOSS 2007 allows global security policies that bind policy settings such as full access, full read access, deny-write access, or deny-all access to a specific user or group within the application. These policies can override the granular permission settings provided by MOSS and managed from the SharePoint Central Administration interface, so you'll want to plan your configuration carefully.

Bringing this all together, your externally facing site might have two URLs as defined in your AAM settings: one for your corporate users to enter, and a second for external users. Each has a unique URL-perhaps contoso for internal users, and extranet.contoso.com for trusted external partners. While both of these URLs will map to the same content, each can access the site through its own zone, with its own authentication provider and its own application security policy. Configuring the intranet zone on the internal URL lets internal users resolve to their domain-authenticated Windows identities while external partners log in via Web single sign-on authentication through the extranet zone, which allows them to use their own organization's user credentials. By leveraging the power of binding Web application security policies to zones, it becomes possible to give all trusted external users full-read access, as opposed to manually setting it on an object-by-object basis.

5. Targeted Content Locking down your SharePoint site collections means users will only handle information you authorize them to access. New flexible permission mechanisms allow you greater control over the various types of business information stored in your MOSS environment.

MOSS 2007 gives you the option to bind an identity to a specific object, whether a site collection, an individual site, a document library, a subfolder, a list, or an individual document or list item. This enforces granular access controls and explicit membership to that item, both denying access and adjusting the user interface so that users are unaware of items they don't have access to. This functionality in MOSS is known as Item Level Security (ILS) or Secured Objects (SO).

6. Integrating Pluggable Single Sign-On The MOSS SSO service provides an encrypted back-end cache of users' credentials for mapping to connected line-of-business systems. This can aid in the retrieval of business-critical information to display through MOSS mechanisms such as the Business Data Catalog (BDC) or SharePoint DataView Web Parts (DVWP).

The MOSS SSO provider, like several of the other application architectural features discussed, is pluggable; you can specify a unique SSO provider outside of the one shipped with MOSS (SpsSsoProvider), though only one SSO provider per line-of-business system can be registered at a time within the environment.

© Copyright 2007, Tomas Elfving

Yahoo joins OpenID initiative

2008-01-21T08:00:00.000+01:00

OpenID is an open SSO-architecture for external sites that makes it unnecessary for site owners to implement their own Identity Management solution. It works pretty much like Microsoft Live (formerly Passport), but with the significant difference that with OpenID there are many "publishers" of login accounts. Any site can start issueing OpenID accounts. For instance, Yahoo announced recently that they are joining the party adding their 248 millions of users to the crowd enabling them to instantly log into all other sites using OpenID. Microsoft and Google has announced that they are backing OpenID, and it appears OpenID has the potantial of becoming the de facto standard for internet SSO during 2008.

Wanna try it out? There is an excellent step-by-step guide at:
http://www.plaxo.com/api/openid_recipe. At http://blog.tomaselfving.com/2007/12/openid-20-released.html there is an list of libraries with APIs encapsulating the OpenID APIs making the implementation even easier.

© Copyright 2007, Tomas Elfving

MOSS Audiences with SharePoint Groups and AD Groups

2007-12-16T07:12:00.000+01:00

I’ve done a number of posts on Audience targeting, and many people have commented or e-mailed me with issues regarding using SharePoint Group based on AD groups. They typically want to use target audience at a web part level using SharePoint Group based on AD groups, without import anything in MOSS repository. This works for AD users, but not for AD Groups. Similar problems are also described in posts like http://www.eggheadcafe.com/software/aspnet/30055451/sharepoint-2007--audienc.aspx ,
and in http://objectmix.com/sharepoint/297158-sharepoint-2007-audience-targeting-domain-groups.html.

It is obviously some kind of issue here, Christopher White seems to be onto something, he says:

“By using the SPUser.Groups property you can easily enumerate the groups that a user has been assigned to. However one problem with this approach is that if the user is a member of a domain group that has been allocated to a SharePoint group, then this group does not appear in SPUser.Groups.”

So, if the SP Groups audience targeting mechanism were based on the ".Groups" property then this could be the problem. But only MS can know this...

One reader have opened an incident to Microsoft on this. I will keep you informed.

Inkblot service, OpenID and SSO

2007-12-10T09:54:00.001+01:00

Didn't really explain the link between the inkblot service and the OpenID SSO solution in my previous post. The real benefit is linking it with Web-based single sign-on (SSO). You create one really strong password using the inkblots and use it to log into an OpenID provider. The OpenID provider then validates your authentication to any OpenID-compliant site on the Web where you have a password-protected account. That means you don' t have to create a password for each Web site you visit because the Web site trusts the OpenID provider to do that authentication. So the benefit is you don't have all those passwords for different sites. You don't have to do the "remember password" thing and then have the password in clear text stored in your in-box folder. Of course, the one password system requires the user have a strong password and this is where the inkblot method comes in handy. It is a single point of failure. If someone cracks that password they can get into all your accounts. For this combination of technologies to fly , OpenID has to become more widely accepted. It is picking up steam, Microsoft supports it in CardSpace, and version 2.0 of Open ID was just released and together with a productified inkblot service, it has the potential to be a both secure and userfriendly service.

© Copyright 2007, Tomas Elfving

Microsoft Research releases the Inkblot service

2007-12-08T08:55:00.000+01:00

Microsoft Research has as a part of its backing of the OpenID standard released an experimental provider. It's basic idea is as follows:

You will be presented with a series of inkblot pics. Think of a description for each inkblot, then type the first and last letters of that description. For example, if the inkblot makes you think of "cloud" then enter "cd". Use a singular term, since it's hard to remember whether you were thinking of "car" or "cars".

Every time you type two characters, you'll advance to the next inkblot, until you have produced a quite difficult-to-guess password. Enter the two-letter descriptions again in the second box. Here, the inkblots appear in a different order, to encourage you to use the inkblots to build associations.

Whenever you are asked to log in, you will be presented with the inkblots to remind you of your associations. After a few logins, you'll no longer even need to look at them: you'll have memorized a strong, difficult to guess password.

Try it out on http://www.inkblotpassword.com !

© Copyright 2007, Tomas Elfving

OpenID 2.0 released

2007-12-08T07:46:00.000+01:00

The OpenID 2.0 (or to be precise OpenID Authentication 2.0 and OpenID Attribute Exchange 1.0) was released a few days ago on the Internet Identity Workshop in Mountain View CA. Both specifications have evolved through extensive community participation and feedback and each have been stable for a number of months.
Both Microsoft and Google shipped OpenID features in beta products. Microsoft Research announced the Inkblot Service as an an experimental Provider while Google announced the ability to comment on Blogger blogs using OpenID.
There are a number of of open source libraries out there including Google’s Blogger (via Sxip’s library) and Drupal who did their own implementation of the specifications. Multiple OpenID Providers including MyOpenID, Sxipper, and VeriSign’s PIP already have support for both of these specifications. Today the following libraries exist which implement OpenID Authentication 1.1 and 2.0, OpenID Attribute Exchange 1.0, and OpenID Simple Registration 1.0:

This SSO standard is definately any public site developer should consider.

© Copyright 2007, Tomas Elfving

Sorting out WSRP, JSR168 and Web parts

2007-11-20T06:27:00.000+01:00

I’ve noticed a lot of confusion and misunderstandings around these portlet ”standards”. This posting aims to sort things out on a fairly high level of understanding by reasoning around the strengths and weaknesses and suggesting some best practices.

So, what is WSRP?

The abbreviation stands for Web Services for Remote Portlets, and the key word here is “Remote”. It is now a fairly old standard, version 1 came 2003 and version 2 is still in draft although many portal vendoers have already made their own implementations of the not-decided version 2. The standardisation work is run in an OASIS committe’ (http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsrp).

WSRP basically defines how to use SOAP to request and retrieve blocks of HTML, instead of XML as regular web services does. This means that the WSRP portlet runs on a one web server and can appear in one or many web portals running on other web servers by sending HTML-code to these web portals. WSRP portlets can be written in any programming language. Before getting into the reasoning around when to use WSRP, we need to sort out the two “non-remote” portlet specs, JSR168 and Web parts.

Comparing JSR 168 portlets with Web parts

JSR 168 is a standard for coding portlets in Java. It consists of a set of class definitions to be installed on portal products supporting JSR168.

To compare apples with apples, JSR168 competes with Web Parts. WSRP is a completely other kind of fruit. Both can support WSRP. These are the most important points of comparison between Web parts and JSR 168:

Language : JSR168 portlets can only be developed in Java. No other language works. Web Parts can on the contrary be developed in any language supporting .NET, which at least in theory includes Java/J++…
Support for different portal plattforms : JSR168 portlets can, at least in theory, be codes once, and run in any os the JSR168-compliant portal produkcts out there. In reality, it is very easy in the Java development tools for each portal product, include product-specific class libraries breaking the standard. Using them makes You tied to a proprietary portal product anyway. In any case, every portlet needs careful testing and debugging when porting from one portal product to another. I personally think that the JSR168 in reality provide very little value from a standardization och reuse point of view. Web Parts makes no secret it’s a proprietary technology and Microsoft makes the most of that freedom not having to compromise in standards committees. Web parts is therefore a superior portlet technology to the JSR168 implementations.
Support for different environments : JSR168 portlets are just portlets. They only execute in the context of at portal product. Web Parts runs in MOSS/Sharepoint Portal, in Windows as Win32-apps as well as in so called Smart clients for handheld devices.
Consuming WSRP : Some portal products have built-in WSRP support while others goes through a JSR168 portlet. Microsoft takes that road by offering at out-of-the box WSRP consumer Web part.
Producing WSRP-portlets : Most JSR168-portal products includes code to emit JSR168-portlets as WSRP. Microsoft haven’t built in WSRP produces support in MOSS 2007, but 3rd-party solutions from NetUnity Software supplies tool to develop WSRP producer portlets on the .NET platform.

This is a simple architecture for JSR168/Web parts portlet applications:

When to use WSRP portlets?

Scenarios motivating use of WSRP are:

From a host / web server deliver portlets as presentation oriented web services to be included in aggregating web porta application. On such host can cater for many web apps.
Web portal apps consuming and in a portal framework integrates presentation oriented web services delivered by internal and external WSRP suppliers.

The WSRP-specification says nothing about the implementation of the portlet, it defines its interface. This is where WSRP connects to Java's portlet specification, JSR 168, and Microsoft’s Web parts. WSRP is not a competing technology, it’s more like complementary. JSR 168/Web parts are used to define the portlet whereas WSRP is used to define the portlets appearance in a remote portlet container (like a web portal app).

Experiences of WSRP

WSRP has some limitations when it comes to complex web applications with advanced user interfaces (interactivity, design). These limitations are:

Separation and handling of presentation/design and the use of Cascading Style Sheets.
Handling of DOCTYPE making accessibility /WAI-adoption harder
A WSRP-portlet knows nothing about its content. The content and design of a WSRP portlet may look different from the rest of the web page it is appearing in.

How to think

If You have a ”pure” Java or .NET –miljö without any needs to remotely distribute portlets, then You develop directly in JSR168 or Web parts. No need for WSRP.

The advantages of WSRP are obvious, it can easily be used in any portal application without local installation, but what are the downsides?

Consider if You can live with the limitations described above.
The extra development work to add the WSRP-interface to Your portletspå dina portlets.
Code running locally (JSR168 and Web parts) generally runs faster than remotely run code. There is overhead in the Web service calls.
Code running locally has access to more information about the execution environment, both the hardware to execute on and the web page to appear in. Using that information, the portlet can act smarter. Communication with other portlets are easier if you know what other portlets that surrounds You.

One smart way of setting up a WSRP-enabled system architecture is to use a “portal UI staging tier”:

The plan here is to develop WSRP producer portlets either in in Java or .NET. Render them in a web portal application using a thin layer of portal specific code (for instance a JSR168 or Web part). This is a SOA-friendly architecture, easy to deploy and modify, that also scales well.

© Copyright 2007, Tomas Elfving

BEA turns down Oracle bid

2007-11-17T22:36:00.000+01:00

It appears my guest appearance as IT business analyst wasn't completely out-of-line as BEA on Friday turned down Oracle $6,7 billion bid, saying it seriously undervalued the company. Now I'm only waiting for HP to make its move ;-)

© Copyright 2007, Tomas Elfving

Wouldn't HP be better for BEA than Oracle?

2007-11-03T09:23:00.000+01:00

I generally don’t comment on IT business acquisitions and business events but this one has implications on the entire application/portal-server market, so I feel it is time for an exception ;-).

The backgroud is Oracle’s recent bid to aquire BEA Systems.

Oracle feels they needs to fill out its product portfolio by acquiring BEA, but will the acquisition bid be successful? Oracle will be faced with a great deal of redundancy in the product range, and as fierce as the competition in that portal servers market is, it is hard to believe that any player has the resources to R&D more than one portal server platform.

Futhermore, It won't be easy to convince BEA's customer base that Oracle only has their best interests at heart. BEA is a serious competitor to Oracle, with some fine technology that is making it difficult for Oracle to become dominant in the middleware space - so BEA has to go whatever the consequences.

BEA has made it clear that being acquired by Oracle is not foremost in its business plan, but Oracle several times has shown its ability aquire companies against their will. It therefore looks likely that BEA will ultimately be acquired - but not necessarily by Oracle. BEA has a strong product set, but would benefit from the additional financial stability of being acquired by a large IT vendor. If not Oracle, then which other vendors might join in the chase?

IBM would be the most obvious competition for Oracle, and it has also been very active on the acquisition trail recently. However, the same issue of product set overlap would make this equally unwelcome to BEA's customers and board.

However, BEA has been very successful in partnering with HP, and since HP abandoned its service-oriented architecture (SOA) platform foray (which it entered through the earlier acquisition of Bluestone) the two product sets have become extremely complementary. HP has waded into the SOA management and governance market through the acquisition of Mercury, but has steered clear of providing its own SOA platform. The combination of HP and BEA, if executed well, would create a significant force in the industry and a natural balance to the otherwise dominant IBM and Oracle.

If BEA should happen to survive the onslaught by Oracle, it would certainly be weakened by the experience and will probably see its market share slip. This would not be good for customers, shareholders or employees, and would probably lead to a further acquisition attempt (by Oracle or others) being ultimately successful. Now might be a better time for BEA to seek a friendly acquisition by a vendor looking for something more positive than simply to trash the competition.

Popfly, an enterprise-ready environment for end-user-driven mashups?

2007-11-03T07:39:00.000+01:00

Popfly features programmable blocks using JavaScript is at the moment not very integrated with the .NET framework-based technologies. Popfly is an example of evolving technologies that exploit the power of combining mashups and Web 2.0.

Initially, the Popfly environment will mostly target public Web users, but it has the potential to become an enterprise-ready environment for end-user-driven mashups. By “Enterprise ready”, I mean that the Popfly environment and block portfolio is managed as an enterprise repository containing assets to enable end-user composite application building. The potential benefits are compelling and with Popfly, Microsoft joins product such as Yahoo (Pipes), IBM (QEDwiki), Oracle (WebCenter) and BEA Systems (Aqualogic Ensemble) in the mashup tool market.

Popfly is primarily interesting in two ways:

The Silverlight-based user interface is powerful, engaging, and sufficiently easy to use that non-developer users can get involved in building composite applications. Users make mashup applications by usings the build-in blocks library, or build their own blocks. These blocks can be service-oriented architecture (SOA) services. The blocks may include visualization service blocks or data service blocks, and the application can also demonstrate a block that handles transformation between two other blocks. Popfly goes far beyond dropping a gadget into a Web portal.
Popfly is also a mashup Web 2.0 community, applying the "network effect" to the mashup world. A user-driven mashup composite application environment is only as valuable as the quality and quantity of services (or blocks) available to users. Users can think of the Popfly community as a "programmable Web" that allows developers and nondevelopers to easily find, consume, create, rate, sell and share blocks and mashups. Popfly's success on the public Web will depend on Microsoft's ability to engage and develop this community.

Popfly now in public beta

2007-11-03T07:36:00.000+01:00

Microsoft is now done ironing out the bugs and feedback gathered during the invited-only alpha release period. Now anyone with a Live ID can start a Popfly account and start being creative.

Microsoft® Popfly™ is a web site and tool to help people create and share web sites, mashups, and other kinds of experiences. It has two parts: the social network ("Popfly Space") and the online tool for creating different mashups and web sites ("Popfly Creator."). In addition to the Popfly web site, Microsoft® Popfly™ Explorer Beta is an add-on component for Microsoft Visual Studio® 2005 that enables you to create and share Visual Studio® projects on the Popfly Space network.

The list of added features of the beta can be summarized as follows:

Gadgets. Popfly can create both Windows Vista Sidebar gadgets and Windows Live gadgets.
Tweaking and Properties. Last iteration, we added “tweaking,” but many blocks didn’t have a lot of properties to tweak, so we added properties to a host of our output blocks including Photoshow, Virtual Earth, PhotoSphere, Gauge, and Page Turner.
Tweaking and the color picker. When you tweak a mashup you now have a color picker so you don’t have to remember the alphanumeric codes for colors.
New and updated blocks. Popfly has a category for new and updated blocks.
New screencasts. Updated videos explaining how to use Popfly.
New blocks.
Updated block documentation.

Finally, check out the release notes before you get started!

Microsoft announces new identity model in Sharepoint

2007-10-17T05:29:00.000+02:00

Microsoft is now making an interesting move as they announce to replace the authentication system for SharePoint Server. Microsoft plan to make the collaboration platform one of the first of the company’s marquee applications to rely on a new claims-based identity model.

The goal is to have SharePoint incorporate an authentication model that works with any corporate identity system, including Active Directory, LDAPv3-based directories, application-specific databases and new user-centric identity models, such as LiveID, OpenID and InfoCard systems, including Microsoft’s CardSpace and Novell’s Digital Me.

SharePoint will lose its rigid authentication system and replace it with an claims-based authentifiation solution. Claims could for instance be age or group membership, that are passed to obtain access to the SharePoint environment and to systems integrated with that environment. Claims are a set of statements that identify a user and provide specific information.The claims are used by systems to make such decisions as who gets access, who can retrieve content or who can complete transactions.

“We don’t want to come up with another, or the next, authentication system for SharePoint,” says Venkey Veeraraghavan, senior program manager lead for Office SharePoint Server.

Veeraraghavan said Microsoft settled on a claims-based system because it is flexible and designed for heterogeneous identity environments. “It allowed us to invest in one place [SharePoint] and know that we can credibly say we work with multiple systems, especially as they are woven into what we’re calling a Metasystem. We want to continue to work on making SharePoint useful to our customers, not spend a lot of time integrating with each and every identity system one-by-one, or worse, not do it because of resource concerns.”

In its current release SharePoint is fairly limited with authentication mechanisms. You can use NTLM (ancient and inefficient), Basic (used with SSL and the clear-text passwords are SO not good) or Kerberos (complex to configure, but better performance) or use MS Single Sign-On. This new move sounds like a great way to open up their collaboration platform to third party options, which are what most companies use.

Kim Cameron, Microsoft’s identity architect, believes an industry transformation to claims-based identity is 18 to 24 months away, which would, considering the normal product release cycle of the Office platform, place the implementation of new claims-based identity model in the next major release of Sharepoint Server 2009 or later.

Read full article

© Copyright 2007, Tomas Elfving

Security and identity in Mashup applications

2007-10-13T08:58:00.000+02:00

The most interesting mashup apps will be creations powered with our personal and business information. So far, there haven't been a single-sign-on model supporting the mashup programming model. Most mashups we've seen so far don't require (or support) logins that allow it to collect information from your private repositories of information. While initiatives like OpenID have the potential to resolve some of these issues, there is a lot of work to be done before the average user will trust a mashup with access to their private information. In coming posts, I'll dive into the Open ID initiative to find out its potential to fill this important technology gap that fundamentally limits the real value that mashups have the potential to provide.

© Copyright 2007, Tomas Elfving

Problem seeing updates in AD groups in MOSS solved

2007-10-10T22:22:00.000+02:00

There have been comments regarding a problem of synchronizing AD and Sharepoint groups ("Audience Targeting and User Profiles in MOSS 2007") . Tim O finally figured the solution and here's his description:
"If experiencing problems seeing updates to the AD groups, it's possible the timer service has stopped and failed to restart. A few people at MS are aware of this problem. In Central Administration for SharePoint, select 'Operations', then 'Timer Job Status' to view the most recent timer job activity. Several jobs are visible here and some run infrequently.

On your SP server, in the services section, restart the following service:
Windows Sharepoint Services timer."

Thanks Tim O for sharing!

© Copyright 2007, Tomas Elfving

Components of an Enterprise Search system

2007-10-09T22:33:00.001+02:00

In a previous posting, I described a custom-made enterprise search solution. This time I look into the characteristics of generic search systems that are available on the market. In the decision between developing a custom solution or buy a commercial system, an important aspect will be the amount of work you need to put in to make the commercial product actually do what you want. This article aims to give You a better understanding of what factors that affect the feasibility of these products.

The components and the behaviour of an Enterprise Search System
Search systems act in two directions: toward the content and toward the searcher. Search engines indexes content to enable accurately sorted result sets and process queries, to put it simply. To accomplish these tasks, most search systems consists of three major quite interdependent components:

1. Content acquisition
2. Indexing service
3. Query processing (including parsing, matching and post-processing)

If something goes wrong in any one of these subsystems, the problem can significantly downgrade the performance and effectiveness of the search engine as a whole. If the content acquisition function is not able to identify and incorporate new content in the index, then the index will obviously soon become outdated. If heavy work load overloads the query processing system, the resources needed to perform index updating to refresh the index may not be sufficient, or the response time for displaying result sets to users degrades to an unacceptable level.
Performance is important an issue in all parts of an search engine solutions. Most of the processes in a search system are processor- and disc-intensive. In order to improve the overall performance, additional hardware, memory, storage, or bandwidth may be needed. To speed up response time in the query processing module, one might need to limit the number of users, place a ceiling on number of results displayed per query, or eliminate certain resource hungry indexing processes such as automated metatagging.

Content Acquisition
You need a plan how to aquire content into Your search solution. The following strategies may be used separately, or mixed. There is no single strategy that always works for everyone:

1. The content acquisition subsystem gathers content to a common index database. Content must be identified, then copied from its location to a processing folder, and finally moved to a “to be indexed” folder. This is the approach most enterprise search systems use.

2. The second way is ton set up the servers with content to run a script and identify changed or new content, then copies that content to a new folder, maybe do some processes the content before moving the processed files to the indexing subsystem. This is an approach supported by such enterprise systems as FAST Search& Transfer, Verity, and others. This makes it possible to plan when updates of the index occurs to avoid performance-heavy indexing updates when a lot of users do searches.

3. Finally one can use so called “spiders”. A spider is a script that on a scheduled basis visits servers, folders, or files. When a change or a new document is identified, the script copies the file to the index processing subsystem. This is an approach supported by virtually all search systems today, but remains a surprisingly complicated exercise. Spiders can for instance experience problems with session variables in URLs, JavaScript, Flash, and forms. When improperly configured, a spider can chase its own tail through a series of infinitely recursive links.

These three content acquisition techniques can be used separately or together by the search system. When the enterprise search system offers application programming interfaces or toolkit modules, highly customized content acquisition systems can be developed. For example, mission-critical content data can be acquired on a near real time basis and incorporated into th search system using scripts whereas non-critical may be acquired on a longed interval. No single content acquisition approach is appropriate in most organizations, hybrid content acquisition techniques are the norm.

To improve content acquisition, a search system can integrate with a records management system, a document management system, or other enterprise information management system that holds information in a particular format or structure. This type of integration typically requires custom scripting and may require additional infrastructure modifications. Similarly, compliance with Basel II, Sarbanes-Oxley or other mandated guidelines will require customization of the content acquisition module as well as other parts of the search system to ensure that versions of documents are not made unfindable.

Indexing Service
The goal of storing content data in an index is to optimize the speed of finding relevant documents for a search query. No one really want a full “byte-by-byte” disk scan for every search query. It may take minutes or hours to complete for an index of say 10 000 documents (which is really nothing in most search solutions). With an index we talking a few milleseconds for the same operation!

To provide a set of matching items quickly, a search engine will typically collect information, or metadata, about the group of items under consideration beforehand. For example, a library search engine may determine the author of each book automatically and add the author name to a description of each book. Users can then search for books by the author's name. Other metadata in this example might include the book title, the number of pages in the book, the date it was published, and so forth.

Index Design Factors
The challenge is to find a optimal Index design and it boils down to considering the following index design factors:

Merge factors – It is about how data enters the index, or how words or subject features are added to the index. Do You have multiple indexes that needs to be merged? Can it be done asynchronously? Search engine index merging is similar in concept to the SQL Merge command and other merge algorithms.

Storage techniques - How to store the index data - should the information be data compressed or filtered?

Index size - How much computer storage is required to support the index

Lookup speed – What are Your lookup speed performance requirements? How quickly an entry in a data structure can be found, versus how quickly it can be updated or removed?

Maintenance – How to work with maintenance of the index over time

Fault tolerance - How important it is for the service to be reliable, how to deal with index corruption, whether bad data can be treated in isolation, dealing with bad hardware, partition (database)|partitioning schemes.

Expressing queryies
Search engines provide an interface to a group of items that enables users to specify criteria about an item of interest and have the engine find the matching items within the group. Most commonly, items are documents or web pages and the criteria are words or concepts that the documents may contain.
There are several varieties of syntax in which a search engine user can express a query. Some methods are formalized and require a strict, logical and algebraic syntax. Other approaches are less strict and allow for a less defined query. One form of a less-restricted query syntax is referred to as Natural Language Search, which is a term typically used to describe web search engines that apply natural language processing of some form. For example, instead of searching for one or two words, a query could consist of an English sentence or paragraph. A natural language search engine will then parse the query into words and evaluate searches for these words. This places less burden on the search engine user to formulate a specific query using restrictive, and sometimes difficult to learn, syntax. A second definition of natural language search engines reflects how the search engine performs indexing, unrelated to the query syntax. This requires a semantic understanding of the query in order to disambiguate the text.

Ranking search result sets
A Boolean search for an item within a group of items will either return the exact matching item or nothing. This is a rather orthodox search method where the equality between the desired item and the actual item must be exact. In application, it is sometimes far more beneficial and useful to incorporate a more lax measure of similarity between the desired item(s) and the items that exist in the group being searched. For example, instead of finding only the exact book in a library, a library search engine may return a list of 'similar' books, with the exact book listed first.
The list of items that meet the criteria specified by the query are typically sorted, or ranked, in some regard so as to place the most 'relevant' items first. Placing the most relevant items first reduces the time required by users to determine whether one or more of the resulting items are sufficiently similar to the query. It has become common knowledge through the use of Web search engines that the further down the list of matching items you browse, the less relevant the items become.

© Copyright 2007, Tomas Elfving

An Enterprise Search solution using Lucene

2007-09-20T11:11:00.000+02:00

I recently designed an enterprise search solution for a +50000 employees organization in the public sector. I have layed out the challenges that we aimed to handle in a previous posting (http://tomaselfving.blogspot.com/2007/09/challenges-of-enterprise-search.html). One additional design objective was to minimize the number of synchronous calls to the base systems.

The solution consists of a 5 layered architecture:

Web application layer – any number of web sites and client applications using the search web services of layer 2.
Web service layer publishing web services with customized search functionality.
Data warehouse layer – The index database that the web services performs its search queries in.
Integration layer with components managing the import and the processing of base system data.
Base systems connected to the solution.

Web application layer
The web application’s consists of a single search field on the intranet portal home page and a set of underlying pages displaying the search result sets and other user interactions from the result set pages. It uses three web services exposing the enterprise search functionality found in the web services layer. The result set can easily be merged and sorted together with search results from searches on web sites and document stores.

Web service layer
The web services are designed to expose custom search functionality based on the structure of the data in the base systems. The first WS is the search service returning a result set with the search result, sorted by relevance and ready to display. As the base systems that we have integrated is document management systems, we need a way to retrieve the physical documents. The second and the third WS is to fetch the document from the two base systems. It takes the document id as in-parameter and returns the corresponding document from the base system.

Intermediate database layer and the integrations
The idea is to gather as much information from the base systems in an intermediate database as possible, in order make it available for high-performance search queries. No round-trips to the base systems are required. This way, the base systems are protected against the hard-to-estimate performance load from web applications using the web services. The only disadvantage is that data in the intermediate database will not be as fresh as the data in the base systems. In this case, this is acceptable. Synchronizations may be done as often as you like, we use an interval of 1 hour. The length of the interval depends on:

How long time the synchronization process takes. The interval shouldn’t, obviously, be shorter as the previous synchronization process needs to finish before the next one starts!
How much changes it is in the base data. The more changes, the shorter interval is recommended.

We populate the index using a Microsoft Biztalk orchestration reading in-files from a folder. The two participating bas systems sends their respective XML-files in their own XML format. They contains both metadata and document information for new and changed documents to this folder using FTP. This mechanism ensures that documents are only sent once. We use different XML mapping schemas for each system to translate words with different meaning and perform simple logic processing like formatting for instance.
The information is stored in the index database and free-text indexing is immediately performed on the incoming data. The index database may also store the entire source document content, if You want to make it searchable. This is a figure showing the connection between the web services, the index database and the base systems.

The communication between the base systems works according to the principles of loosely coupled systems. Dependencies regarding system up-time are minimized. With this solution the responsibility to produce and deliver the incoming XML-file including…

filtering of information that the base system doesn’t want to publish and make available for search
and optimization of the data for improved searchability

…lies in the respective base system.

One risk is that the FTP-file transfer fails for some reason, but from a search user perspective the solution still works. The latest data will not be in the index database, but the search function will work. The next time the FTP-transfer works, the index database will be up-to-date again.
One requirement on the base systems and their transfer programs is that they set time-stamps on changes in the base systems. Using that timestamp, only the changed and new data since the last run can be extracted in a batchprogram.

What is this magic Lucene index database then? Lucene is a free/open source information retrieval library, originally implemented in Java by Doug Cutting. It is supported by the Apache Software Foundation and is released under the Apache Software License. Lucene has been ported to programming languages including Delphi, Perl, C#, C++, Python, Ruby and PHP. Our solution used the C# version known as Lucene.NET.
While suitable for any application which requires full text indexing and searching capability, Lucene has been widely recognized for its utility in the implementation of Internet search engines and local, single-site searching. Lucene itself is just an indexing and search library and does not contain crawling and HTML parsing functionality. The Apache project Nutch is based on Lucene and provides this functionality; the Apache project Solr is a fully-featured search server based on Lucene.
At the core of Lucene's logical architecture is a notion of a document containing fields of text. This flexibility allows Lucene's API to be agnostic of file format. Text from PDFs, HTML, Microsoft Word documents, as well as many others can all be indexed so long as their textual information can be extracted.

This is just one way to implement this functionality. It proved to work very well for us, the quality of the search result sets are good, and the performance in term of search response times is fantastic.

Feel free to comment on this or let me know if you have made other experiences in the field of Enterprise search!

© Copyright 2007, Tomas Elfving

Challenges of an Enterprise Search implementation

2007-09-08T08:11:00.000+02:00

Enterprise Search is, to quote Wikipedia, "the practice of identifying and enabling specific content across the enterprise to be indexed, searched, and displayed to authorized users". The goal is to give the users the "single search"-field while still search all kinds of content from all kind of data sources. The challenge is that "content" comes in many formats and from different kinds of data sources. It may for instance be:
- other web internal sites
- your own extranets & internet sites
- file shares
- customer records held in a CRM system
- business information in an internal database
- letters and reports in Document Management Systems.
- people/contact information in a telephony system

Another challenge is that the information often have various levels of security classification so that only authorized user should get hits on a search. Sounds faily simple, right. But that means firstly that the user making the search needs to identify himself, and secondly that all the systems needs to be able to identify that user correctly. Not an easy task when different base systems have their own user database with their own user identification solution. Across an enterprise, numerous user databases may be in use.

Information in base systems is not structured in a "search-friendly" format. A relational database is pretty useless as it is when it comes to extracting relevance-sorted search results from a free text search. You probably need to do some work to make these system available as a good data source in an enterprise search solution.

How do You create a search result when hits comes from many different data sources? In what order should the search hits be sorted? You need a way to understand the relevance weight of each search hit from each data source when assembly the total, final search result to be presented to the user.

Lastly, performance issues needs to be adressed. The users of an enterprise search expects response times of no more than a few seconds, they are spoiled with the Google performance of "0,047 seconds" ;-). The base systems participating in the search solutions are rarely prepared for this scenario.

In an upcoming post I'll describe an enterprise search solution atchitecture that I recently implemented for a large organisation adressing the challenges described above.

© Copyright 2007, Tomas Elfving

Scoping content with Audience targeting in MOSS 2007

2007-09-03T11:18:00.000+02:00

I'm getting comments & questions about different "group problems" on a previous post on Audience targeting ("Audience targeting and User profiles in MOSS 2007"), and it seem to be a good idea to clarify the different options available with audiences. These are some additional thoughts on scoping content in MOSS 2007.

SharePoint Groups - SharePoint Groups are a valid Target Audience mechanism. This is especially useful when the site administrators may not have access to Active Directory. With Sharepoint Groups, the site administrator have full control to modify his audience. SharePoint Groups have the added benefit of allowing self-enrollment if the site administrator wants to setup a site that might have different levels of information and allow the users themselves to subscribe to what components they'd like.
Active Directory Domain Groups or Roles- Active Directory domain groups are also a valid Target Audience. The nice thing with this option is that many organizations already have groups setup for internal use that are suitable for targeting specific areas of an organization. The SharePoint site administrator has however less or no control over the membership in the group. This might, on the other hand, be just what you want!
Audience Rules - These are very powerful and potentially maybe least understood. They can be used to do a number of convenient things. They can be setup with multiple rules and then setup to require a match to all rules or any rule. This allows a SharePoint Shared Services administrator to define and scope very flexible audiences that will update automatically as user information is changed and synchronized in to SharePoint. The rules might be as simple as belonging to a security group or being a part of a specific area of the Actove Directory organizational hierarchy. Group, list and organizational rules have operators of "Reports Under" and "Member Of". User profile property rules have operators of "Contains" and "Not Contains". You could for instance make a basic rule to match all users with the word Sales rep in their TitleSharePoint under their Skills and Manager in their TitleNew Employee Orientation Team distribution list.

SOA defined

2007-08-30T14:37:00.001+02:00

We can define Service-Oriented Architecture as an architectural style for building systems based on interacting coarse grained autonomous components called services. Each service expose functionality and behavior through contracts. Contracts are composed of messages at discoverable addresses called Interfaces.

Service. A Service should provide a high cohesion and distinct function. Services should be coarse grained pieces of logic. A Service should implement at least all the functionality promised by the contracts it exposes. One of the characteristics of services is service autonomy. Autonomy means the services should be self-sufficient, at least to some extent, and manifest self healing properties.

Contract. The collection of all the messages supported by the Service is collectively known as the service's contract.

Interface. The Interface is a URI, a specific place where the service can be found and consumed.

Messages. The communication mechanism in SOA is the message. Messages can for instance be

http GET messages
SOAP messages
JMS messages
and even SMTP messages

Messages must have both a header and a body. The header is usually more generic and can be understood by infrastructure and framework components without understanding, and consequently coupling to, every message type.

Service Consumer. This is the user of the Service. A service consumer is any software that interacts with a service by exchanging messages with the service. Consumers can be either client applications or other "neighboring" services their only requirement is that they bind to an SOA contract.

New to WSS 3.0 development?

2007-08-13T23:46:00.000+02:00

Patrick Tisseghem has written a great introduction to Windows SharePoint Services 3.0 outlining the differences between traditional ASP.NET and WSS 3.0 development. He explains the required development environment, and the steps to build a Windows SharePoint Services solution with Visual Studio 2005 Extensions for Windows SharePoint Services 3.0.
Enjoy step 1 on http://msdn2.microsoft.com/en-us/library/bb530302.aspx and step 2 on http://msdn2.microsoft.com/en-us/library/bb530301.aspx!

© Copyright 2007, Tomas Elfving

MOSS Audiences aren't security objects

2007-08-09T00:26:00.000+02:00

Don't use MOSS 2007 audiences as security objects. They are to be used for audience targeting only. You can target any list item or web part directly to a rule based audience. This enables rule based audiences to be created and maintained in the Shared service provider, while letting an individual site owner use these defined audiences on individuel sites.

Audiences are a global concept, whereas MOSS(Sharepoint) Groups are a relatively local concept. You might reuse the same audience across multiple portals, which will each have their own groups. There is sadly no way to define an audience rule based on groups. On most of the places where you use audience targeting, you can also use SharePoint Groups to target audiences.

Audiences are not used to assign rights and permissions. MOSS uses site groups to assign rights and permissions to users within the portal. Audiences are used to manage how content is distributed, not to enforce security. They push information to a user, not restrict or permit access to information.

Popfly Blocks intro

2007-07-06T06:44:00.000+02:00

In Popfly, You can either create your own web pages, create mashups (A mashup is a web application that combines information from two or more websites or sources.) based on existing "blocks", or create your own blocks. A block is a component acting like a piece of middleware. It is contained in a single JavaScript file (.js), which provides methods for user generated code to invoke. A block may also make use of resource files such as XAML files, images, etc.

A block can act as a middleman between externally provisioned services such as web services, or it may simply be a library of reuseable functions, e.g. a function that convert between currencies or is retrieving customer data from an ERP system. A block can also have a user interface acting as a display surface: something which takes data from other blocks by using their functions and displays it in a meaningful manner, and allows the user to interact with it.

At the moment You can't edit JavaScript directly inte the Popfly environment as the JavaScript editor currently lacks some basic functionality such as syntax highlighting (The Popfly team is working on that.) Use for instance the Microsoft Visual Web Developer 2005 Express, freely available from: http://msdn.microsoft.com/vstudio/express/vwd/ to develop Your Popfly blocks!

© Copyright 2007, Tomas Elfving

Becoming a Popfly user

2007-06-12T06:06:00.000+02:00

The day after I requested an invitation på the Popfly Alpha, I got an invitation by "Suzanne" on the Microsoft Popfly Team. Acception the invitation and it's time to get off to create my own profile with a Display Name, Page Title, Display Title, Tag Line, RSS feed and banner picture. This isn't MySpace, right?

© Copyright 2007, Tomas Elfving

What is Popfly?

2007-06-11T06:44:00.001+02:00

There's a lot of talk about Popfly now. It's a new web application to create Mashups. Mashups is a fusion of information and application based on various web-based sources. A Mashup may for instance link new, video, photo, maps etc together in an integrated application. In Popfly You can easily build and share mashups, gadgets, Web pages, and applications. Popfly consists of two parts:
1. Popfly Creator is a set of online visual tools for building Web pages and mashups.
2. Popfly Space is an online community of creators where you can host, share, rate, comment and even remix creations from other Popfly users.

Check out the Popfly webcast and You will get the big picture.

Popfly builds of the new Microsoft technology Silverlight, which is a crossplatform browser plug-in. A a first glance it seems similar to Flash, but I would not take that comparison any further at this point. There's also a webcast out there about how to integrate Popfly and MOSS: Popfly-SharePoint integration.

I'm requesting an invitation to the Popfly alpha at www. popfly.com. I'm now waiting impatiently...

© Copyright 2007, Tomas Elfving

Evaluating EpiServer 4.61 as portal framework

2007-06-03T06:57:00.000+02:00

EPiServer has since version 4.51 a built-in support for portal functionality through its CMS add-on “Portlet Framework”, and in the latest version 4.61 has the support been further developed. Version 5 is scheduled during the second half of 2007. The Portal framework of EPiServer is based on Microsoft ASP.NET 1.1 and has recently added support for producing and consuming WSRP (Web Services for Remote Portlets) -based portlets and for Microsoft WebParts, also used in MOSS 2007.

Based on the identified evaluation areas described in my previous post "The need for a portal architecture", EPiServer have been evaluated as portal framework for development of both intra-, extra- and internet solutions for larger companies or organizations. The figure below illustrates a fairly generic system architecture with portal services. One important aspect to look into regarding portal products is to what extent the portal services are reusable, programmable components.

Regarding portal services included in the product, EPiServer has the following features.
- Document management
EPiServer has a basic support for document management with capacity to define meta-data and have version management on documents using what EPiServer calls Unified File System (UFS). UFS is a component framework for EPiServer with possibility to extend it with other external data sources. The functionality is fairly basic especially compared to products like BEA WebLogic, IBM Websphere and MOSS 2007.

- Collaboration tools
EPiServer has basic support for collaboration through a predeveloped template where users can share documents, new, calendars and notes. It uses the built-in user- and group management facility of EpiServer.

- Search
The standard installation of EPiServer has a simple search engile covering content stored in the EpiServer database (no external sites can be indexed for search). EPiServer relies on 3rd-party products adding various functionality. SiteSeeker and MondoSearch are two examples of partner products that has proven to work well.

- Workflow
EPiServer has a proprietary framework for configuration and development of workflow functionality using the Windows Workflow Foundation in .NET 3.0.
.
- Forms
EPiServer has forms management based on XForms. Via EPiServers forms management any editor can publish simple forms using the EPiServer editor user interface. For more advanced forms with integration to external systems custom development is required.

EPiServers support for portlets can be used in two ways:
1. Use single portlets integrate with other standard EPiServer modules/controls. This can be done with the standard EPiServer licence.
2. Use the full EPiServer Portal Framework where a page i entirely built by several portlets layed out on the page. EPiServer Portal Framework is an addon product to EPiServer.

Key functions for EPiServer as portal platform:
- Support for WSRP (open standard from OASIS)
- Support for WebParts (Microsoft)
- Support for both produce and consume WSRP-based portlets
- Built-in support for content management
- By supporting WSRP, ability to consume both java and .NET portlets.
- Ability to mix WSRP-portlets with standard controls in .NET and EPiServer

Furthermore one can have personal setting managing which portlets to show and if you want, let the user decide for himself using the built-in administration interface. There are also built-in portlet adapters for publishing content via RSS and IFRAME.

A new version of EPiServer is coming during fall 2007. The new version of EPiServer is launched with a number of architecturally important mechanisms. It will for instance have full support for Microsoft .NET 3.0 including Windows Workflow Foundation and Windows Communication Foundation. EPiServer will also feature a more open programming interface for consumption of components (such as user management) to ensure improved scalability. With these changes EPiServer have taken a big step towards being a more complete platform for development of complex web solutions.

Pros
I have found the following main arguments for EPiServer as a portal platform:

- It has a strong track record for internet sites with portal functionality.

- EPiServer has with its portal framework managed well to add portal functionality with its popular CMS. It supports portal standards well and works fine together with EPiServers regular content management functionality. It is, especially from an editor and administrator usability perspective, an advantage that CMS and portal functionality is handled within the same tool.

- In a scenario where EpiServer coexists with MOSS or Websphere or other portal platforms, its integration capabilities becomes important. EPiServer has support for WebParts, WSRP, IFrames and RSS integration with other portal tools.

Cons
The following weaker areas have been identified with EPiServer 4.61 and EPiServer portal framework:

- Scalability - Version 4.61 EPiServer has a known limitation in handling a large number of registered users (for instance on a intranet or extranet with 10000+ registered users). Even though an external Active directory is used, EpiServer also stores the users temporarily internally. This leads to problems with large number of users when EPiServer does a very performace-heavy search (read JOIN) when authentificating in the EPiServer-database.

- Portlet technology - Just as all products supporting the WSRP, I have seen that the WSRP-technology is still not mature to handle complex web solutions with high requirements om interactivity, design and usability. The most notable limitations are:

Handling of presentation/design and the use of Cascading Style Sheets.

Handling of DOCTYPE which makes accessibility adoption harder

WSRP includes no quality assurance of content. Content in the WSRP portlet can break accessibility compliance of a whole page in the portal

- Search
The built-in search functionality of EPiServer can’t handle the requirements of indexing all the different sites that most companies has. There are 3rd party products such as MondoSearch and SiteSeeker that have been proven successful complements. They can be used at an additional licencing cost.

- Portal components
EpiServer’s support for important components such as document management, workflow, collaboration is rudimentary and insufficient, especially for intranet and the extranet solutions.

Summary
EPiServer is a strong CMS product, it’s ease of use is well documented. As a portal framework however, EPiServer still can’t compete with MOSS and Websphere mainly due to limited functionality in the portal components. On sites with requirements on compliance to accessibility standards and guidelines (WCAG) is EPiServer on the other hand a far better product that both MOSS and Websphere.

The need for a Portal architecture

2007-05-14T11:12:00.000+02:00

On almost any company’s agenda thesedays is the need to increase flexibility and speed of development for digital services. They are the main tool for implementing business processes meeting ever-changing customer demands. The need for efficient content management have been a base requirement for many years now, but to really unleash the potential of digital services, You need an efficient delivery platform. These are commonly called web portals.

A portal is a point for the person gathering of information and consumption of services in different situations. They provide an excellent way for enterprises to provide a consistent look and feel with access control and procedures for multiple applications, which otherwise would have been different entities altogether.

In choosing a portal platform, it is important to study to what extent the portal products supports the integration of digital services (a k a web applications or simply portlets). Portlets are pluggable user interface components that are managed and displayed in a web portal. Portlets produce fragments of markup code that are aggregated into a portal page. These portlets will typically be targeted to different kind of users both internally, among partners and externally (see previous post on Audience Targeting ).

Typically, following the desktop metaphor, a portal page is displayed as a collection of non-overlapping portlet windows, where each portlet window displays a portlet. Hence a portlet (or collection of portlets) resembles a web-based application that is hosted in a portal. Portlet applications may generic portlets such as include email, weather reports, discussion forums, news and business specific portlets like product search & sales, support issue management.

Portlets may be used as user interface components implementing efficient business processes providing the users a great service experience. Portlet standards are intended to enable software developers to create portlets that can be plugged in any portal supporting the standards.

The new thing is that the business process and the collaboration around these processes, not the web content, is at the heart of the new generation of portal solutions. Important supporting technologies include workflow management (business process automation), document management, forms management, system integration, collaboration, personalization, multi languages and security.

In evaluating portal products, I suggest the following focus areas of investigation:
- Portal application development and portlets
The portal application development and portlets verification aim to investigate the efficiency and ease of portlet development using the recommended portlet standard (web parts, WSRP, JSR168). The distribution of functionality between CMS-tools and portaltools may be a problematic area, especially from a content editor/administrator perspective.

- Interfaces and principles for integration
Evaluating integration options to existing platfirms and systems in the IT infrastructure. Look into how well the products expose their components according to a service-oriented architecture.

- Included portal components
Investigate the level of functionality of the out-of-the box portal components. Document management, collaboration, workflow, search and forms management is particularily interesting.

- Web standards
How well will portal solutions build on each tool conform to accessibility standards such as WCAG (Web Content Accessibility Guidelines)? WCAG is based on specifications from W3C (World Wide Web Consortium) and is aimed for everyone developing content and services for web publication.

The following figure illustrates the necessary components of a portal platform and the composition of these components.

I will deconstruct this portal architecture in great detail in an upcoming series of posts.

© Copyright 2007, Tomas Elfving

Audience Targeting and User profiles in MOSS 2007

2007-05-02T05:13:00.000+02:00

The concept of Audience Targeting brings a new dimension to web portal site design. You are not any more restricted to define a single (or maybe two; the unknown's and the logged-in's) target audience, with targets audiences the granularity of user group definition has decreased significantly. You may now distinguish one user from another based on any single piece of information that you have on that user, and customize the services and content presented to that user by dynamically showing relevant Web parts based on the values of the Web part's custom properties. An audience can be identified by using SharePoint groups based on Active Directory, distribution lists, or security groups or by using a rules-based system to create a global audience.

By using Audience Targeting in MOSS 2007, You can define an audience by specifying criteria to define a subset of your user crowd. Once you have defined an audience, you can then configure content such as list or library items, navigation links, and Web Parts to conditionally display its content only when the current user is a member of that audience. This makes it straightforward to target services and content to those users who need it while hiding that content from users who would find it distracting. On an intra-or extranet these groups may be defined as an Active Directory user group like "Project managers" having for project managers customized Web parts presented to them. You can also use custom properties to define audiences for users that You don't store in the Active Directory. It could, for global audiences on an internet eCommerce site, be that You want to give special attention or service on your site to customers that have previously spent more than a certain amount. You don't need to be logged in, the custom properties of a Web Part could be set by using session variables from user interaction, or read from a cookie.

Audience Targeting builds upon another nice feature of MOSS, User profiling. Within an organization, there is generally a lot of information about a user stored in various IT systems. A user has a network login, an EmployeeID, name, an address, a phone number, a title, maybe a photograph, current assignments, time reporting information and the payroll department may have tax and salary information.

This information usually is stored in disparate systems that aren't integrated with each other. The User profiling feature provides a solution by making it possible for all this information to be stored in one system, so that specific content could be targeted to specific users. So, for example, news regarding a project could be viewed only bu people assigned to work on that project. User profiles allow you to associate metadata with every UserID. This metadata can then be kept in sync with the other systems in the organization using the Business Data Catalogue connections or an Active Directory.

User profile has, among others, the following features:

Multivalued properties - Properties may have multiple choice list values, for instance for a "Competence" property.
Properties with open or closed choice lists - You can tie user profile properties to a vocabulary constraining the list of possible values. The list can be open so that users can add new choices, or closed so only administrators can edit choices.
Property mapping from external data sources other than Active Directory, such as an entity registered in the Business Data Catalogue.
Property privacy policies restricting who can access the property. You can set policies to restrict access to Only Me, My Manager, My Workgroup, My Colleagues, or Everyone, and to enforce that a property is required, optional, or disabled.

User data can be imported into a User profile from these data sources:

Any catalogue directory service supporting LDAP (Lightweight Directory Access Protocol), for instance Novell eDirectory, IBM iSeries Directory Service etc.
Databases
Enterprise applications (such as SAP or PeopleSoft)

Active Directory is not required, but still the recommended technology for user data storage for MOSS.

On the Microsoft Sharepoint Products and Technologies' blog, they have provided step-by-step instructions on setting up personilazation.

ASP.NET 2.0 CSS Friendly Control Adapter to solve the accessibility issue in MOSS

2007-04-28T06:31:00.000+02:00

The problems in MOSS regarding accessibility and web standards is getting more and more attention (see also http://tomaselfving.blogspot.com/2007/04/web-parts-and-accessibility-in-moss.html). Frank van Rooijen addresses the issue and points at a one solution based on the Microsoft ASP.NET 2.0 CSS Friendly Control Adapters in his blog http://blogs.tamtam.nl/frank/PermaLink,guid,aad39854-4f7d-4666-8a0a-0fc02f3cd01b.aspx

Copyright 2007, Tomas Elfving

Web 2.0 and the future of desktop apps

2007-04-20T05:34:00.000+02:00

Web 2.0 has entered the business main stream. For a phrase coined by O'Reilly Media only in 2004, referring to a perceived second-generation of Web-based services that emphasize online collaboration and sharing among users, Web 2.0 seems to have already established itself as a driving force amazingly quickly, but appears to mean somewhat different things depending who you ask.

By using the version-numbers concept commonly used for software upgrades, the phrase "Web 2.0" confusingly hints at an improved version of the World Wide Web; it is instead name for web-based applications with a rich user interface often featuring collaborative functionality. It is built upon technologies (that have been around since long before the "Web 2.0"-expression buzz began) such as weblogs, social bookmarking, wikis, podcasts, RSS feeds , social software, Web APIs, Web standards and online Web services.

Time bar of Web 2.0 buzz words. This image shows the age of some buzzwords sometimes used in Web 2.0 lingo and its dependencies. (picture from http://en.wikipedia.org/wiki/Web_2.0)

Web 2.0 brings with it is a shift from desktop applications to shared spaces — collaboration delivered as a service over the internet. This is a break with the desktop centric world we have lived with during the "Windows" era. Rather than working out of a desktop application and exchange dokuments in various formats and expecting the reciever to have the same software installed to be able to read and work with the document, we may collaborate in the same "document" using shared spaces (wikis, blogs, project rooms, blogs, …) with invited friends and colleagues. Both long-term cooperation needs and more sponataneous situations is catered for. A new project may be initiated with an online project management tool, bringing a virtual team together across physical locations, time zones, and even organisations to work on a common project. The shared space space will just as efficiently be used for a few hours to resolve an emergency issue. Web 2.0 has obvious attractions in the enterprise, providing flexibility and improved cooperation by quickly being able to bring the right people to every task collaborating across space, time and room.

Still the Office-packages installed locally on the users desktop dominates . When it comes to document sharing and collaboration, portal tools with collaboration functionality only goes as far as making it possible to share document files in workgroup spaces.

Things will, however, become really interesingly when the real Web 2.0 killer app makes is launched with full strength: The web-based online word processor and spread sheet (see http://docs.google.com, formerly www.writely.com, for a tryout of this technology). Having the functionality of letting many users work together in the physically same document and store the document directy on internet file shares, combined with the power of collaboration enabling teams to collect, share and discuss these documents and information withing workgroups, I believe the use of desktop Office application suites slowly but inevitably will decrease.

Web Parts accessibility limitations in MOSS

2007-04-11T13:30:00.000+02:00

The support of accessibility (WCAG 1.0) in Web parts on pages in MOSS is limited. Stay with ASP.NET controls and leave the Web parts until your users really need that personalization functionality. When they do you can add personalization support and start using the WebPartZones in MOSS, and accept the accessibility limitation. This article outlines these limitations and discusses some consequences.

The WCAG standard

The W3C standard “Web Content Accessibility Guidelines 1.0 (WCAG 1.0)” ( http://www.w3.org/TR/WCAG10 ) covers a wide range of issues and recommendations for making Web content more accessible. The document contains principles, guidelines, and success criteria that define and explain the requirements for making Web-based information and applications accessible. "Accessible" means usable to a wide range of people with disabilities, including blindness and low vision, deafness and hearing loss, learning difficulties, cognitive limitations, limited movement, speech difficulties, photosensitivity and combinations of these. Following these guidelines will also make your Web content more accessible to the vast majority of users, including older users. It will also enable people to access Web content using many different devices - including a wide variety of assistive technologies.

What is Web Parts?

Web Parts are server-controls (portlets) that can be placed on a web page where the end-user is given the possibility to edit content, appearance and behaviour of the portlet directly from within the web browser. By integrating Web Parts into web pages using WebPartZones and WebPartPages in MOSS, the end-user can also add, remove, move around, maximize and minimize WebPart-portlets. When the end-user make changes to these portlets or pages, the changes can be saved and reused in later browser sessions.

The big advantage with web parts is that it gives the end-user the possibility to personalize the content of a web site dynamically without the involvement of an administrator or a developer. It is also possible to import and export personalized WebPart-settings between different pages and sites. ASP.NET 2.0 has built-in support for WebParts. Basically, WebParts in ASP.NET are common server controls, but with the difference that it has support for personalization through the WebPart-interface. To get the full potential out of Web parts (that is, the personilazation functionality), the Web parts one must put the in a WebPartZone using a WebPartManager store and retrieve personalization data to a database (a MS SQL Server or Sharepoint Server) is also required. One can also put a common ASP.NET server control in a WebPartZone. These control will in that case be wrapped in a GenericWebPart-contol.

Accessibility limitations when using Web Parts

The implementation of WebPartZones in MOSS is hardcoded (unlike the corresponding classes in IBM Websphere. They are just as poorly implemented, but at least open to rewrite as you please) and doesn't support the accessibility guidelines on a number of instances:

WebPartZones uses HTML tables for its graphical design whitch breaks the 5.3 guideline in WCAG 1.0.

WebPartZones does only support only personalization if You use javascript, which breaks guideline 6.3 in WCAG 1.0.

At the time of writing, the rich user interface of Web parts only works with Microsofts own web browsers:
- WIN IE 6.0 and above

It doesn’t work very well in the following browsers:
- WIN Firefox 1.5 and above
- WIN Opera 9 and above
- MAC Safari 2.0 and above
- MAC Firefox 1.5 and above
- MAC Camino 1.0 and above
- Linux Firefox 1.5 and above
- Linux Konqueror 3.4 and above

This breaks the guideline 6.4 in WCAG 1.0. This means that pages that uses web parts will be hard to use in other browsers other than Internet explorer. The pages will not be adapted to mobile user interfaces and devices used by for instance vision imparired people.

A WebPart can also run outside MOSS and works in this scenario like any other ASP.NET server control. It may not rely on personalized data. I can, however, not see any advantage in developing controls as Web parts instead as web controls unless they are going to be used as personalizable portlets in WebPartZones.

PS. This article partly builds upon the work of Martin Odhelius.