Category Archives: Domino

06.12.18
by

Domino 10 vs NoSQL

By Tim Davis – Director of Development.

With Domino 10 bringing Node.js, and my experience of  Javascript stacks over the past few years, I am very excited about the opportunities this brings for both building new apps and extending existing ones. I plan to talk about Node in future blogs, and I am giving an introduction to Node.js at MWLUG/CollabSphere 2018 in Ann Arbor, Michigan in July.  However, I would like to digress from the main topic of Node and Domino itself, and talk a little about an awareness side effect that I am hoping this new feature will have, i.e. moving Domino into the Javascript full stack development space.

There are a plethora of NoSQL database products. In theory, Domino could always have been a NoSQL database layer, but there was no real reason for any Javascript stack developer to even consider it. It would never appear in any suggested lists or articles, and would require some work to provide an appropriate API.

The thing is, working in the Javascript stack world, I was made very aware that pretty much all the available NoSQL database products did not appear very sophisticated compared to Domino (or most other major Enterprise databases – Oracle, DB2, SAP, MS-SQL, etc). The emphasis seemed always on ease of use and very simple code capabilities and not much else.

Now in and of itself this is a worthy goal, but it doesn’t take long before you begin to notice the features that are missing. Especially when you compare them to Domino, which can now properly call itself a Javascript stack NoSQL database.

Popular NoSQL databases are MongoDb, Redis, Cassandra, and CouchDb. As with all of the NoSQL databases, each was built to solve a particular problem.

MongoDb is the one you have probably most likely heard of. It is the ‘M’ in the MEAN/MERN stacks. It is very good at scaling and ‘sharding’ which is sharing workload across many servers. It also has a basic replication model for redundancy.

Redis is an open source database whose power is speed. It holds its data in RAM which is super-fast but not so scalable.

Cassandra came from Facebook, and is a kind of mix of table data and NoSQL and is good for very large volumes of data such as IoT stuff.

CouchDb was originally developed by Damian Katz from Lotus and its key feature is replication including to local devices, making it good for mobile/offline solutions. It also has built-in document versioning which improves reliability but can result in large storage requirements.

Each product has its own flavour and would be suited to different applications but there are many key features which Domino provides, that we are used to being able to utilise, and while some of these products may have similar features, none of them have a proper equivalent for all.

Read and Edit Access: Domino has incredibly sophisticated read and edit control, to individual documents and even down to field level. You can provide access through names, groups and roles, and all of this is built-in. In the other products, anything like this has to be pretty much obfuscated by specifying filters in queries. You are effectively rolling your own security. In Domino, if you are a user not in the reader field then you can’t read the document, no matter how you try to access it.

Replication and Clustering: One of Domino’s main strengths has always been its replication and clustering model. Its power and versatility is still unsurpassed. There are some solutions such as MongoDb and CouchDb which have their own replication features and these do provide valuable resilience and/or offline potential but Domino has the most fine control and distributed data capabilities.

Encryption: Domino really does encryption well. Most other NoSQL products do not have any. Some have upgrades or add-on products that provide encryption services to some degree, but certainly none have document-level or field-level encryption. You would have to write your own code to encrypt and decrypt the individual data elements.

Full Text Indexing: Some of the other products such as MongoDb do have a full text index feature, but these tend to be somewhat limited in their search syntax. You can use add-ons which provide good search solutions, such as Solr or Elasticsearch, but Domino has indexing built-in and those indexing solutions themselves have little security.

Other Built-in Services: Domino is not just a database engine. It has other services that extend its reach. It has a mail server, it has an agent manager, it has LDAP, it has DAOS. With the other products you would need to provide your own solution for each of these.

Historically, a big advantage for the other datastores was scalability, but with Domino 10 now supporting databases up to 256Gb this becomes less of an issue.

In general, all the other NoSQL products do have the same main advantage, the one which gave rise to their popularity in the first place, and this is ease of use and implementation. In most cases a developer can spin up a NoSQL database without needing the help of an admin. Putting aside the issue of whether this is actually a good idea for enterprise solutions, with containerization Domino can now be installed just as easily.

I hope this brief overview of the NoSQL world has been helpful. I believe Domino 10 will have a strong offering in a fast growing and popular development space. My dream is that at some point, Domino becomes known as a full stack datastore, and because of its full and well-rounded feature set, new developers in startups looking for database solutions will choose it, and CIOs in large enterpises with established Domino app suites will approve further investment in the platform.

05.31.18
by

Are You Ready? System Requirements #Perfect10

In this 4th webcast in my #Perfect10 series I discuss system requirements for v10 of Domino, Sametime and Traveler.  Yes I know we don’t know those yet and we don’t even have the beta but we do know some things that are coming and more importantly this is something you should do before any major upgrade regardless.  If we want an upgrade to be successful we don’t want it dragged down by old or outdated architecture and operating systems.

It runs a little bit longer than I like at 19 minutes I had a lot of information to cram in. I’m sure you can speed me up to 1.5x if you want to save a few minutes 🙂 As always if you have any feedback or would like me to do a webcast on a specific aspect please let me know.

05.30.18
by

What is NoSQL?

By Tim Davis – Director of Development.

I have been working with the MEAN/MERN stacks for a few years and with Domino 10 looking to introduce Node.js support, Domino itself is following me into the ‘World of Node’. This world is the full-stack web developer world of MEAN, MERN, and all things javascript, and in this world NoSQL is king.

The MEAN/MERN development stacks have been around for a while. They stand for MongoDb, Express, Angular/React, and Node. (The other main web development stack is LAMP which is Linux, Apache, MySQL, PHP/Perl/Python).

The reason the MEAN/MERN stacks have become so popular is because they are all based on the same language, i.e. javascript, and they all use JSON to hold data and pass it between each layer. It’s JSON all the way down.

You may already be using Angular or React as a front end in your Domino web applications. With the introduction of Node into the Domino ecosystem, this becomes even more powerful. Domino can become the NoSQL database layer of a full javascript stack (e.g. DEAN or NERD) and, most importantly in my view, Domino becomes a direct competitor to the existing NoSQL data stores such as Mongo and Couch which are so popular with web developers and CIOs.

So what exactly is NoSQL?

As you can tell by the name, it is not SQL. SQL datastores are traditional relational databases and are made up of tables of data which are indexed and can be queried using the SQL syntax. Examples are DB2, Oracle, and MySQL. The tables are made up of rows with fixed columns and all records in a table hold the same fields.

NoSQL data is not held in tables. It is held in individual documents which can each hold any number of different fields of different sizes. You can query these documents to produce collections of documents which you can then work with.

Does this sound familiar? Yes, this is exactly how Domino works! Domino was NoSQL before NoSQL.

The main advantage of NoSQL over SQL in app development is that it allows for more flexibility in data structures, either as requirements evolve or as your project scales. It also allows for something called denormalization, which is where you hold common data in each document rather than doing SQL joins between different tables, and this can make for very efficient queries. Again, this is how Domino has always worked. Notes views are essentially NoSQL queries.

In addition to all this, when NoSQL is used in a javascript development stack the use of JSON as the data format means that the data does not need to be reformatted as it passes up and down the stack, with less chance of errors occurring.

Now obviously the note items inside Domino documents aren’t held as JSON, and this would be a issue when looking to integrate Domino into a javascript stack, but the Node server solution being introduced in Domino 10 solves this problem.

The Node server in Domino 10 comes with a ‘connector’ to do the work of talking to Domino. It is based on Loopback and gRPC (both IBM/HCL initiatives) and promises to be very fast. Having this connector built-in means that you as the developer do not need to worry about how to get data out of Domino. HCL have done that job for you. All you have to worry about is what to do with it once you have it, e.g. send it out as a response from an API, show it in Angular or React, or whatever.

This is all very exciting as a developer, especially one like me who has worked with javascript stacks for a while, but as I mentioned earlier the power of this solution is that it moves Domino into a position to directly compete with other NoSQL databases in IT strategies.

In my next blog I will talk about the advantages that Domino brings to the NoSQL table and why I believe it is the best NoSQL solution for full-stack javascript development.

05.24.18
by

Engage Week & Lots of News

This week was the Engage conference held in Rotterdam – the largest and (IMO) best event Theo Heselmans has given us yet.  Rotterdam is a lovely city and the water taxi that took us from the restaurant back to the boat last night turned a 5 minute ride into a James Bond chase sequence – at several points he took corners by tilting the boat almost entirely on its side (there goes Tim!) and then onto the other side (bye Mike!) before pulling a handrake turn and reversing up to the dock – worth every cent of four and a half Euro.   I don’t usually find time to attend sessions beyond the keynotes because I get caught up presenting and doing other things (I find it hard to think what right now but let’s group it under “meeting people”) but this week I was rushing from presentation to round table to meetup so here’s a summary of my highlights, kept as short as I can so you aren’t tempted to tl:dr

HCL brought the energy, the enthusiasm and a huge team of people showing how far they have taken Domino, Notes, Traveler, Sametime , Verse on Premises etc.  IBM had energy too but their focus was Connections/Workspace and although it continues to develop, we in the ICS community have been starved for progress on the other products.  HCL together with IBM hosted several roundtables on Domino, Application Development, Notes Client, Verse on Premises etc where we got to ask for or complain about what we wanted or felt was missing and answer questions about design priorities.  I won’t go through all of that other than to apologise to everyone else in the Domino/Sametime roundtable who didn’t get a word in once I started.

From that Domino round table we heard about a couple of much needed and unexpected features coming in v10 (both of which I think are so new they haven’t yet been named) around the area of TCO. One is what I’d call a sync feature for Domino where you can tell a server to keep specific folders in sync with other servers in its cluster. Those folders could contain NSFs but also NLOs (DAOS files), HTML files or really anything else.  The server will create the missing files and it doesn’t use replication to do that.  Even better, if the server detects a NSF file corruption it is capable of removing its own instance of the file and pulling a new one from a cluster mate – all without any admin intervention.  Another great tool will be the idea of shared encryption keys for NLO files so that Server B will be able to copy even encrypted NLO files from Server A by decrypting and then re-encrypting them.  Management and maintenance of NLOs and the DAOS catalog was high on my list of enhancement requests.

From the Application round table we heard about how the integration with Node and Domino will work,  there will be a npm install – DominoDB that will allow Node developers to access Domino data via the Node front end. Queries to Domino from the Node server will be using high performance gRPC (remote procedure calls)  – in the same way Notes and Domino use NRPC for proprietary access. The gRPC access used by Node for Domino will eventually be open source.  The front end of the Node server will be surfaced using the Loopback API gateway.

Essentially what this means is that any developer who can program using Node will be able to use their existing skills against Domino NSFs.  That Domino systems will, in one step, become accessible to a much wider group of developers and systems is the main application development goal.

Domino statistics and reporting can be uploaded into and analysed from within the New Relic platform.  If you find this as interesting as I do then you too are clearly an administrator,

HCL Places.  So that was a surprise.  HCL demoed a working (but very basic) prototype of a new product they had been developing in secret (well no-one in the room knew of it).  A lightweight desktop collaboration client that runs against a Domino NSF. It can include mail,, sametime , video, mentions and Notes applications.  All on premises.  Here is a terrible image of the prototype which – yes I know is cluttered – but focus on the features not the look and you can see that HCL are trying to take Domino somewhere we’ve all known it could go but never had the chance.   The image was shared out by Jason Roy Gary who built and demonstrated the prototype and whose role at HCL is (I think)  Vice President Engineering and Innovation, Collaborative Workflow Platforms.

Dd4EspOUwAAwlFb.jpg-large

In a week full of good news the two best were that a beta program for v10 will start with phase 1 in June and phase 2 in July.  June will be a closed beta and July open.  If you want to register for the beta program when it is announced then sign up for the newsletter on the Destination Domino site here

Plus there was this .

IMG_0018

I don’t want to minimise the contribution by IBM themselves at Engage, each of the roundtables included IBM’ers alongside HCL’ers and there was certainly plenty of activity around Connections and Workplace but right now, in this blog, I’m revelling in the fact that Domino is finally getting the attention it deserves.   Plus look at these great pens – they have little yellow highlighters in the top and when I asked IBM if I could buy some for customers they were happy to give me a “few”.

IMG_0126

So – long story (it could have been sooo much longer) short.  A great week , I learnt a lot, my session on Docker was standing room only in boiling heat, I had the chance to talk to people I rarely get to talk to and Engage was in another great location.  I don’t know how Theo will match this next year but I look forward to finding out.  Plus I got chocolate as a speaker gift.

Now don’t go messing with my high.

05.22.18
by

V10 Roadmap: What’s new in Mail, Chat, and Verse on Premises?

Following on from our presentation at IBM Think, on Thursday May 24th I will be presenting a follow up webcast with Ram Krishnamurthy, Chief Architect, Notes, Designer and Xpages (HCL) and Andrew Manby, Director of Project Management (IBM).    On the webcast we will be showing the latest additions to Notes client mail, calendaring and Verse on Premises all of which comes from live code and will ship with v10 of each product later this year.

If you saw our presentation at Think there have been more additions and changes since then – the speed at which the products are being developed is something I haven’t seen before and there are some great new features and UI changes I think you will like.

We have a lot of content to cover in 45 minutes and Andrew will have some news you will want to hear too so go here to register for the webcast starting 10am EST, this Thursday the 24th of May.

If you want to stay up to date with all the changes happening to Domino, Sametime, Traveler, Verse and other products then keep an eye on the Destination Domino site where all the news and announcements appear first, and while you’re there why ot sign up for the newsletter.

As we all get ready for v10 of the products later this year I will be blogging more of my own preparation work on my blog at https://turtleblog.info and also populating a Youtube playlist called Perfect10 with a series of 10’ish minute videos to help you prepare.

05.15.18
by

Are You Ready: Domino #Perfect10

In today’s edition of my #Perfect10 webcast I discuss some steps in finding your Domino servers, reviewing their dependencies and auditing database access.   This is a 15 minute presentation which I’ve tried just as slides with my voice instead of video.

As always please let me know what you think and anything else you would find useful.

Next Up: Are you ready – Traveler, Sametime and Web Mail

 

05.10.18
by

Path To A Perfect 10 #perfect10

Here’s an intro to a new blog series I am starting to hopefully take us all through preparing for v10 of the ICS products due out later this year.  If you have suggestions for what you’d find useful outside of the things I mention in the video please let me know.  I’ll be using the hashtag #perfect10 for this which I’m sure isn’t used anywhere else and won’t lead to any confusion 🙂

Any feedback as to format is welcome.  Most blogs will have both slide content and video to match as I thought that the most engaging.  I am trying this as a YouTube channel.

 

05.02.18
by

Destination Domino (yes, yes I’m late to the party) **

Well this is a lot of good news all at once.  IBM have launched the Destination Domino site – a one stop shop for all your Domino v10 and future strategy news.  If you doubt their commitment to the future development of Domino and the community that believes in it, well just look at all that yellow.

On Thursday 24th May (the day after Engage) I’ll be participating in a webcast on what’s new for Mail, Verse, and Chat for v10.  I will be joining Andrew Manby (Director of Product Management @ IBM) and Ram Krishnamurthy (Chief Architect, Notes, Designer and Xpages @ HCL) on the call.  Registration is here. I recommend you also sign up to the newsletter on the Destination Domino site to stay on top of the developments happening because those are coming at us pretty fast.

I was fortunate enough to visit HCL’s offices in Chelmsford, MA last week and met many of the development teams working on Domino, Verse, Traveler and Sametime.  Some I have known from their years at IBM before they moved to HCL and some were new to me – most of the day is under NDA and you’ll be hearing more about what they are going to deliver at some point if you attend Engage, DNUG, MWLUG and other conferences this summer. If you can’t attend just keep an eye on the Destination Domino site.

One thing I can share that isn’t under NDA is how impressed I was not just with the rapid development of features many of us have been waiting a long time for but also the innovative and open thinking behind Domino as a development platform and the energy and enthusiasm just about everyone I met that day (over 30 people) had.  We are going to see a lot more on the Notes client for iPad and the integration of Node.JS in the next few months and that’s all very exciting.

**I have a good excuse since I’m currently on holiday in St Lucia BUT we interrupt this pool / beach time because this is really important.

03.01.18
by

My Favourite Things From Yesterday’s Call – What’s Coming …

Yesterday IBM and HCL hosted a call introducing some of the changes on the way with Domino, Sametime, VOP, etc, as a result of the worldwide Domino Jam sessions they have been running this year.  I attended an earlier Domino Jam in London in January which was jointly hosted by IBM and HCL and was very pleasantly surprised by how receptive they were to really unusual ideas as well as honest criticism so I was looking forward to hearing the results.

To start with it’s worth knowing that the strategy is continually evolving, what we were told about and what HCL are committing to is just a first step on a multi-year multi-release journey and that’s good news.

  • Modern development tools and frameworks
  • Easier and extended authentication and admin options *
  • Core performance improvements *
  • Slimmer, faster, better looking Client *
  • Adding mail and calendar features
  • Improved integration for mail and productivity with Microsoft Office

The items with asterisks were those I certainly raised (ok, went on and on about) during the Domino Jam and it’s great to see others did too and pushed them onto the priority list. I was also pleased to hear them talk about Domino as the original NoSQL database server which it was and remains – I honestly believe customers need to be reminded that before NoSQL was what everyone was doing, Domino was doing NoSQL 🙂

The application development innovations include support for Node.js integration and the ability to deploy applications on an iPad – we saw some code but it’s early days for a release due in Q4 and involving such huge amounts of work. There was a definite amount of “teasering” (made up word)  going on and a promise of hands-on access to beta code at Think in a few weeks as well as at user groups like SNOUG (in Switzerland), Engage (in Rotterdam) and Collabsphere (in Ann Arbor).  If you can make it to Think there’s going to be a lot to hear about and get your hands on, but if you can’t then there will be continual opportunities throughout the year before everything hopefully ships in Q4.

One very interesting thing that came out is that a lot of enhancements were already developed by IBM but never released into the product (for whatever reason) and HCL have been able to use those to leapfrog forwards.   Things coming to v10 of Domino include:

  • automated database repair (fixup is nice but wouldn’t it be nice if we didn’t have to do that at all)
  • removal of the 64GB limit and a new 256GB limit (which suggests a lot of efficiencies in database structure)
  • investment in Domino docker images to be made available with installs
  • update of SAML (to SAML 4) and improved integration with AD

Obviously this isn’t everything, there are more things appearing every day and much is still under NDA but we’re months away from release and there’s a feeling of a team pushing hard and breaking through old models and definitions to modernise and breathe oxygen into these products.

I have always believed that Domino is the best NoSQL database server and mail server out there and it seems HCL believe that too and are going full out to prove it.  Not to mention some of the most exciting stuff being produced is in the area of application development which I’m not best to comment on but still like what I am hearing.

If you were on the fence about attending Think I would suggest climbing off it – not just so you can benefit from learning whilst there but to grab the multiple opportunities to feed back your own ideas.  There are people behind these products who want to hear them.  If you can’t make Think then sign up for whatever user groups you can.  I will be doing the same.

 

02.13.18
by

Creative Ideas For Docker (and Domino)

In an earlier post I mentioned that I have been working on new technology projects since the end of last year and I wanted to share here what I’m doing as well as plan to keep you updated on my progress if only to keep pressure on myself.   I have been working with, and speaking about, Docker and containers for the past year and it was good news to hear that IBM will now support Docker as a platform for Domino (as of 9.0.1 FP10). http://www-01.ibm.com/support/docview.wss?uid=swg22013200

Good news, but only a first start.  Domino still needs to be installed and run in its entirety inside a container although the data would / could be mapped outside.  Ideally in a microservices model Domino would be componentised and we could have separate containers for the router task, for amgr, for updall, etc, so we could build a server to the exact scale we needed.  However that is maybe in the future, right now there’s a lot we can do and two projects in particular I’m working on to solve existing issues.

Issue 1: A DR-Only Domino Cluster Mate

It’s a common request for me to design a Domino infrastructure that includes clustered servers but with at least one server at a remote location, never to be used unless in a DR situation.  The problem with that in a Domino world is also Domino’s most powerful clustering feature, there is an assumption that if a server is in a cluster then it is equally accessible to the users as any other server in the cluster and, if it’s not busy and the server the user tries to connect to is busy, the user will be pushed to the not-busy server.   That’s fine if all the cluster servers are on equal bandwidth or equally accessible, but a remote DR-only server that should only be accessed in emergency situations should not be part of that failover process.   It’s a double edged sword – we want the DR server to be part of the cluster so it is kept up to date in real time and so users can fail over to it without any configuration changes or action on their part.  We don’t want users failing over to it until we say so.

I tend to tackle this by designing the DR server to have a server_availability_threshold=100 which marks it as “busy” and prevents and client failover if the other servers are online.  It works ‘ish’ but someone has to disable that setting to ensure all users failover neatly when needed and it isn’t unusual to have a few users end up on there regardless.

So what can Docker do for me?

I don’t see that much value in a standard Domino image for docker in my world.  When I build a Domino server it tends to have a unique configuration and set of tasks so although it would be nice, my goal in deploying Domino under docker is very different. It is to create identical containers running identical versions of Domino with identical names e.g Brass/Turtle and Brass/Turtle. Both containers will point to external data stores (either in another container or a file system mount). Both will be part of a larger Domino cluster.  Both will have the same ip address.  Obviously both can’t be online at the same time so one will be online and operating as part of the cluster and only if that server or container goes down would the other container – at another location – activate. In that model we have passive / active DR on a Domino server that participates fully in workload balancing and failover.  I don’t have to worry about tuning the Domino server itself because the remote instance will only be active if the local instance isn’t.   I would use Docker clustering (both swarm and kubernetes can do this) to decide to activate the second container.

In principle I have this designed but I have lots of questions I need to test.  Not least deciding the location of the data.  Having a data container, even a clustered data container would be the simplest method.   That way the Domino container(s) would reference the same data container(s) however Domino is very demanding of disk resources and docker data containers don’t have much in the way of file system protection so I need to test both performance and stability.  This won’t work if the data can be easily corrupted.   The other idea is to have a host-based mount point but of course that could easily become inaccessible to the remote Domino container.  I have a few other things that I am testing but too long to go into in this post.  More on that later.

Issue 2: Domain Keys Indentified Mail for Domino

In its simplest explanation, DKIM requires your sending SMTP server to encrypt part of the message header and have a public key published in your DNS file that enables the receiving server to decrypt it, thereby confirming it did actually originate from your server.  It’s one of the latest attempts to control fraudelent emails and, combined with SPF records, constitutes requirements for DMARC certification.

The DKIM component of DMARC is something Domino does not support either inbound or outbound.  It may do in the future but it doesn’t right now and I am increasingly getting asked for DMARC configurations.  Devices like Barracuda can support inbound DMARC checking but not outbound DMARC encryption. The primary way I recommend doing that now is to deploy Postfix running OpenDKIM as a relay server between Domino and the outside world, your mail can then be “stamped” by that server as it leaves.

My second docker project therefore is to design and publish an image of postfix + OpenDKIM that can be used by Domino (or any SMTP server).

More on these as I progress.