03.30.16
by

Sametime WAS Proxy Stops Working

I’ve had an interesting system down call with an existing Sametime 9.0.1 customer in the past week.  The environment is over 18 months old and consists of every server component in single instances including ST Proxy, Meetings, ST Advanced and all Media components.  The media components were added in Dec 2015 and everything has been fine. The Meeting and Proxy servers both have WAS proxies in front of them to handle traffic over port 80 / 443 separately.  Last week the Meeting node was restarted and the WAS Proxy stopped working.  It would load.  The Meeting server was responding on its own application ports to http(s)://hostname:9080 / 9443 both worked but http(s)://hostname failed with

503 Service Unavailable

The WAS Proxy server showed started.  There were no errors in the logs for that or the ST Meeting server.  Not all WAS proxies were broken because the one in front of the ST Proxy server worked.  In short that error suggests that the Meeting server is offline when we knew it wasn’t and since there isn’t any real configuration for the WAS Proxy other than what node it points to – there was nothing to troubleshoot.  I tried deleting and recreating the WAS Proxy a few times, I tried switching it to use alternate ports 81/444, nothing would fix it.

It took a few days and some combined effort to find.  The WAS team wanted us to upgrade to WAS fixpack 5 but that would mean upgrading 8 working servers in the hopes of fixes one WAS proxy.  There was a suggestion that since the Meeting server was a single, not a cluster, I could just change the Meeting server ports to use 80/443 instead of 9080/9443 and do away with the WAS proxy entirely.  That would get rid of the problem but not fix it, just circumvent it.  I wanted to fix it and find out why it happened.

I had checked the virtual hosts to make sure the hostname / port combination was in the stmeet host and wasn’t anywhere else and discovered that in default_host new wildcard port entries had appeared for ports 80 and 443.  I had already deleted those but that didn’t fix the problem.  How did those port entries appear ? I’ve seen this before when you install new ST servers (as we did with Media in Dec) it come sometimes write virtual host entries to the wrong places.  In fact that was my first guess but after I removed those entries from default_host and it still didn’t fix the problem I was out of ideas.  Then Tony Payne from IBM spotted that the admin_host virtual host which is only used by the SSC had the ports 9080 and 9443 in it when it should only have 8700 and 8701.  Again I assume these were added by the previous server installs and of course I never went to look there because the Meeting server was specifically set to use the STMeet host.

I removed those extra ports from the admin_host virtual host definition and restarted the Meeting node and servers (clearing the temp directories first \profilename\temp and \profilename\wstemp as well as \profilename\config\temp) and that fixed the problem.

So why was the presence of those two ports 9080/9443  (used by the ST Meeting server) that were in a virtual host the ST Meeting server doesn’t even use causing the WAS Proxy to break? Why didn’t the Meeting server itself break and why didn’t the ST Proxy Server which also had a WAS proxy in front of it break?

Turns out that no matter what virtual host mapping you have in place for applications, in Sametime the code checks the admin_host and if a port appears there – it silently disables looking up any other host.  The fact that the Meeting server ports appeared at all in the admin_host meant that the STMeet host was being ignored and the WAS Proxy had no way to direct the traffic.

Unfortunately none of that is visible in the logs or in debug logs which all reported the servers and services using the correct STMeet host.  So it wasn’t something that was able to be seen.  It was a combination of Tony seeing the admin entries and me having had a previous call with a server install which added ports to unwanted virtual hosts that allowed us to find it and fix it.

The ST Proxy server itself wasn’t affected because that server was running on 9082/9445 so its ports weren’t in admin_host and its virtual host therefore wasn’t ignored.

Always good to have a problem fixed and learn a ton of stuff about application behaviour at the same time 🙂

03.30.16
by

Last week in Eindhoven…

We were in Eindhoven last week at the Engage conference.. over 400 attendees, speakers and IBM’ers gathered for two days of learning, talking and cleaning out the hotel bar of tonic water.. I’ve been to several of the past Engage conferences and Theo always puts on a great event but this was bigger and better than ever.  So why?

IBM sent a lot of executives to Engage with the Opening General Session being given by the new ICS general manager (appointed at Connect in January) Inhi Cho Suh and with product strategy presented by Suzanne Livingston , Sara Gibbons and Chris Crummey.  The first thing Inhi announced was that things are going to change – starting with the Orlando conference which moves to February 22nd at Moscone West in San Francisco.  That’s a big decision and commitment – serious tech companies have conferences in SF and that’s where ICS (IBM Collaboration Services) need to be if they are going to innovate, lead and grow as opposed to maintain.   Inhi also let us know that she has asked the product team to work on a 2020 strategy and that it will include IBM Verse on premise.

Then we got the demo of Verse , Toscana and the thinking behind ICS design.  It’s a shame the OGS wasn’t recorded as Suzanne’s background to their design thinking and Sara & Chris’ demo were both much more detailed (and further advanced) than at Connect in January.  However if you want some idea of what we saw take a look at the OGS video from January (from about 90 seconds in to 20 mins in) here

Aside from the OGS the entire IBM team (of which there were more than 30 in attendance) were everywhere wanting to hear about problems, wanting to listen, wanting to change their relationship with partners, with customers with development for the better.   It’s hard not to be taken up with the positivity and enthusiasm.  I’m an optimistic person but I don’t consider myself naive – I feel that I recognise honesty and intent when people talk to me and I what I heard that ICS was important, investable and part of the core IBM development strategy.

In short I choose to believe until I’m proved wrong.

There were of course plenty of great sessions to attend and, as usual, I missed many of the ones I wanted.  Partly because there were also lots of round table discussions too which I found very interesting.  Apparently I’m still the 8 year old in class first to put her hand up with a question.

My session on SHA2 and SSL vulnerabilities was against Mat Newman’s User Blast and Sara Gibbons’ with Toscana.   We were all along the same corridor and I watched person after person go past my room on their way to Mat or Sara’s , so thank you to everyone who chose to hear about security instead and filled out my room.  I hope you found it useful  (and the hand puppets helpful).  For anyone who wasn’t there I have added it to slideshare 

On the final evening of the event Theo invited speakers to a dinner preceded by a surprise.  The surprise was that 32 of us were sent into the Escape Rooms.. you are locked in a themed room for an hour and have to decode lots of puzzles to find the code to get out.  I’ve always wanted to try an Escape Room and I chose the “Tomb” which was an Egyptian tomb and went in with a team including Tim and Mike, Sue Smith, Bill Malchisky, Mat Newman, Rene Winkelmeyer and Carl Tyler.  We didn’t make it out in time – we were soooooo close.. but a few things to bear in mind

  • The tomb was entirely dark except for a small flashlight Tim found hidden in a basket in a corner and some candles.  My night vision varies from “bad” to “crappy”
  • Having multiple alpha males in a small space all shouting instructions at each other may not be the best way to get out quickly
  • There was sand everywhere.  Everywhere.  My shoes may never recover
  • Tim is great at puzzles but apparently in the dark, without his glasses (which he forgot to bring in) and with 7 people shouting at him to hurry up – not so much
  • There was a really cool effect where we completed a puzzle and lasers appeared out of the eyes of a skull on the wall and we had to position 7 different mirrors around the room to bounce the lasers around to hit a small hole on the wall.  We got so excited doing that we didn’t notice we had completed the puzzle and a new “door” had opened for about 10 mins.
  • I was given a cryptex to decode and open.  I broke it by pulling the end off.
  • With only 1 light source we could only do one thing at a time so some of us spent a lot of time kneeling in the sand feeling around fake skeletons for clues

In the end it was great fun and I’d definitely want to do it again.

All of that plus a chance to talk to lots of customers and see lots of friends – some of which came along just to meet up.

I hope you’re recovered Theo – because we’re all up to do it again next year.

 

 

 

 

 

03.16.16
by

Spark Ideas – Connect 2016

This year at Connect in Orlando the Nerd Girls and IBM once more hosted the Spark Ideas session where we ask people to talk for around 6 mins on any topic they like.

This year we had some great from changing career to changing country, to dealing with large life changes to growing your own vegetables. We managed to get the sessions recorded* and we have now uploaded them into a youtube playlist for you to enjoy.

I hope you find them as entertaining, enlightening and moving as the audience in Florida did.  A huge thank you to Amanda Bauman, Jamie Magee, Ginni Saini, Chris Miller, Lauren Maxwell, Theo Heselmans, Mat Newman and Carolyn Pampino for so generously sharing

*apologies to Mitch Cohen who took the raw video as I then edited them and uploaded them. Any issues with the video are entirely down to me.  Thanks Mitch!

03.14.16
by

Whilst You’ve Not Been Looking Something Amazing…

Next week in Eindhoven there is an IBM conference running for two days but if you haven’t been paying attention you may not have noticed what Engage has become and if you don’t act very quickly  – like today quickly, you’ll miss a fantastic and free opportunity.

Started and run by Theo Heselmans , Engage has grown far beyond the original Belgian community and this year – with almost 400 attendees and 85 speakers he’s providing an event for attendees from around the world and another great location – the Evoluon in Eindhoven.   Even if you made it to Florida in January (and especially if you didn’t) what’s awaiting you in Eindhoven is something very new including strategy sessions, round table discussions and content that hasn’t been seen anywhere else.  That includes my session – this year with puppetry (because how better to explain SSL vulnerabilities than by using hand puppets) .

The keynote is being given by an IBM team led by Inhi Cho Suh the new General Manager for IBM Collaboration Solutions, Suzanne Livingston (Group Product Manager, IBM Mail, Chat, Meetings & Social Cognitive Solutions), Chris Crummey (World Wide Executive Director of Evangelism and Customer Experience) and Sara Gibbons (Senior User Experience Designer).  

The round table discussions  are definitely something I’ll be attending too, scheduled 4x each day and each individually hosted by either someone from IBM or an event sponsor. Want to talk to Inhi Cho Suh about ICS strategy? Ed Brill about Verse Deployment or Matt White about extending your skillset to Cloud technologies?  You won’t easily get that opportunity again.

All that is just the beginning, take a look at the full agenda here then RUN don’t walk to try and grab the last 20 or so places before registration closes.

As for me, I’ll be driving over next Tuesday and presenting Thursday morning so I have plenty of time to go to sessions myself and just generally see and talk to people.  If you want to meet up or have any questions to ask me, just email or DM me and we’ll get something organised.

My session is at 11.30 on Thursday morning  is Building & Deploying SHA2 Certificates
If you saw my session with Mark Myers at Connect in January you may wonder if this is the same thing.  It’s not.  For a start I have hand puppets and no Mark.  I hope to explain where to expect your highest risk, how to stay secure by having a strategy and how to generate and actually deploy SHA2 certificates in Domino, WebSphere and IBM HTTP Server.

I hope to see you in Eindhoven !

 

02.23.16
by

Goodbye SocialBizUG & Thanks For Everything

As you’ve probably seen SocialBizUG (http://www.socialbizug.org) has closed down and IBM have launched a new IBM Social Business User Community site.

SocialBizUG was owned and managed by the team at WIS who have been such a large part of our community for many years through the View magazine and conferences, the dev and admin tips newsletters and much more.  They remain very active in the publishing and conference space, just not our little corner of it 🙂

I”d like to send out a big thank you to the WIS team who produced and edited so much great content over the years especially Celia Hamilton, Sarah Cenedella, Stephen Gainer, Sue Morrell, Debbie Lynd and Noreen  Chase.   We live in a big world of technology co-existence – I hope to run across you again soon. 

02.09.16
by

Connect 2016 – A Wrap Up

As I get ready to leave Florida tomorrow I realise that I haven’t blogged since Connect began (for me) on Friday 29th.  This was our 21st IBM Orlando conference and like everyone else I was prepared to see it go forever or merge with Las Vegas.  That hasn’t happened and I hope after the last two years it won’t.

My Connect starts early on Saturday with meetings of The Penumbra Group an organisation of Business Partners who work with IBM social software. We joined Penumbra in 1996 after starting Turtle and realising we were isolated as a small BP in a small corner of a small country, it allowed us to talk openly to and partner with others in a similar situation around the world.  Thanks to Penumbra we met some of the smartest people doing what we do and many of them are now my closest friends.  We start our meetings talking about our businesses and what struck me this year was how technologically diverse everyone has become.  There’s no such thing as working with a single technology anymore, it’s about integration, coexistance and enhancement.  That’s a positive theme I saw repeated throughout the conference.

Monday morning saw a new format Opening General Session. It’s a hard balance to get right with an audience of CIOs, analysts, press, customers, sales people and BPs – at least one or two sections will be boring for someone.  This year IBM gave very clear information in advance on how the OGS would work, an opening hour of strategy and review followed by an hours break and then another hour of demo and customer stories.  I’ll be honest, I doubted how many would return after the break but it’s a testament to how well judged the format was that just about everyone came back.  It was fast paced, well structured and interesting (actually good customer stories).

Looking back 10 years the OGS used to be about underlying technology, we’d hear about changes to Domino view indexes or a new Sametime feature but we’re not there anymore – the underlying architecture and how to discussions are saved for breakout sessions, the OGS is about VISION and I say that with no sarcasm at all.  What I want from IBM in the OGS is what I got – a clear vision of where the technology is going, where my customers need to be,  and what and when IBM plan to deliver along that path.

The less said of the opening speaker the better – a shouty millenial who actually mouthed along to the videos from his youtube channel as they played and I suspect plans to patent the word EXPONENTIAL.  He was no Brian Cox or Neil DeGrasse Tyson but at least he wasn’t around for long and the follow on speakers were all great.

There were lots of good news items and great demos of ongoing developments such as the personal assistant features in the mobile application and the evolution of the Verse UI as well as Project Toscana which is a new persistent chat client that will integrate with Verse, with Connections and with other shared data. Connections itself is being redesigned to ensure you are being shown the most useful information for whatever you are doing in an appealing and clear layout (I realise that sounds a bit vague but once there are good screenshots I”ll share them).  Of course there was also the big news I was beginning to wonder if we’d hear ..

IBM Verse on premise built on top of Domino and due this year

.. I have many customers who aren’t ready to move to the Cloud yet but weren’t sure where Domino was headed or if they should be moving “to Verse” or something else.  IBM is committed to the Domino server (which still beats any other mail server into a cocked hat) with a new UI and deeper integration with Connections, *cough*Sametime*cough*, IBM Docs etc.  If you’re on a Domino platform it’s still a critical part of IBM’s server strategy and underpins many of the “futures” we were being shown.  There was also an announcement of Outlook running on top of Domino later this year so that gives you a lot of choices of client to access your servers Notes, iNotes, Traveler, Verse (desktop and mobile), Connections Mail and Outlook.

By the time the OGS drew to a close the nervous cynicism of Sunday had become tentatively excited optimism. For me that’s down to two things, seeing IBM clearly showing they understand what their customers need and are working to deliver it (as opposed to telling customers what they need) and a much looser , more inclusive attitude.

I had lots of discussions with friends, customers, other BPs and IBMers throughout the week and the general feel was consistently that a community was being rebuilt, that there are people within IBM who recognise the benefits from creating closer relationships with partners and customers. There were clear signals that IBM were listening and adjusting their strategy.  Less mention of “cloud or nothing” and more “this is where we want to take you but we’re not forcing you down a particular path”.  Yes if I’m being cynical I could point out that the detail of how this would all be accomplished was scarce but I can wait for them to share that so long as the final destination is where I want to be.

Under the heading of “pulling the community together” I’d like to say thank you to Amanda Bauman @ IBM who worked so hard to bring “Swolphin Lotusphere” to “Hilton Connect” , even organising a craft beer night , a Karaoke night and getting the Nerd Girls Sparks session on the agenda.

I used to say that I’d leave Florida each year feeling energised, excited, happy and determined to learn new things.  That was definitely true this year.  I love the Hilton location, by far over the Swan/Dolphin (those outdoor firepits for late night chats are much better than the rotunda) and the bar was nicely laid out (if horrifically expensive and that comes from a girl who buys drinks in London bars).

My Connect Words For 2016

Community

Vision

Conversations

Enthusiasm

Firepit

Sunshine

AGAIN!

Nothing is confirmed yet but I hope to see everyone at the end of January 2017 at the Hilton Orlando..

It’s been real 🙂

 

02.02.16
by

IBM Lifetime Champion Award – 

Today during a lunch for IBM Champions, Amanda Bauman announced a new IBM Lifetime Champion award created this year and given to Theo Heselmans and I.  It was a surprise to say the least and the standing ovation my fellow Champions gave us (along with the hugs and handshakes) sent me into shock

2016 marks the 20th year of Turtle and I have loved and still love working with the products and people in this community (call it what you will, you know who you are) for every moment of that time.  I am beyond happy that I have been part of helping it grow and flourish and that I have learnt so much from so many of you.  

I will do my best to keep learning, writing, sharing and working.  For those of you that have come to my presentations, worked with me, taught me, encouraged me, nominated me and become my closest friends

Thank you

Tomorrow I’m on stage at the closing session of IBM Connect.  Deep breath and wish me luck.