WebSocket support in kbmMW #1

Contents

Preface

The next release of kbmMW Enterprise Edition will include several new things and improvements. One of them is full WebSocket support.

WebSocket explained in few words

WebSocket is a “new” way (standardized with an RFC in 2011) to communicate between web browsers and application/web servers, which allows for a compact two way near real time communication, which is not based on a request/response scheme, but on that each side is listening for messages and pushing messages, in many ways similar to what the WIB (Wide Information Bus) is doing.

Compared to the WIB, there is however no ability to subscribe for particular messages, so WebSocket is by default, not a publish/subscribe based duplex communication.

All major web browsers support WebSocket communication. To be able to use WebSocket on the client side, you will need to code in JavaScript or WebAssembly.

Fortunately it is pretty simple to code for WebSocket in Javascript.

WebSocket communication always starts out with regular HTTP communication.

A browser makes a regular HTTP request to a webserver, asking for the connection to be upgraded to WebSocket optionally with various additional wishes for the detailed communication.

The web server responds either by agreeing to one of the wishes requested by the browser, or by rejecting it all together, in which case no WebSocket communication is possible, at least not with the wishes requested by the browser client.

WebSockets in kbmMW context

kbmMW has for many years supported acting like a regular WWW server and as a REST endpoint. This is a requirement for being able to do WebSockets.

In addition kbmMW has usually been using the standard Indy components for transport, although other ways has been made possible over the years.

However kbmMW has for a couple of years included some extremely fast and efficient socket components, which moves kbmMW to the top of the performance board when doing HTTP or REST operations.

Obviously WebSockets continue to build on that technology, rather than on Indy which means you will get a very fast WebSockets able web server if you use kbmMW. Check this performance blog to notice that kbmMW actually beats all other competitors in most comparisons including all the popular variants used by non Delphi developers.

A sample kbmMW based WebSocket server

We make a simple Delphi project. It will contain a form, and an HTTP service unit along with some HTML/Javascript it can serve to a client. The server will provide a rudimentary chat client for browsers.

As you may notice, the sample includes the option to let the Webserver offer access via SSL depending on if the Use SSL checkbox is checked or not.

The Listen button contains this code:

procedure TForm6.btnListenClick(Sender: TObject);
begin
     if FServer.Active then
     begin
          FServer.Active:=false;
          btnListen.Caption:='Listen';
     end
     else
     begin
          FTransport.Host:='0.0.0.0';
          if chbUseSSL.Checked then
          begin
               FTransport.UseSSL:=true;
               FTransport.Port:=443;
               FTransport.SetSSLCertificateFromFile('.\domain.crt');
               FTransport.SetSSLPrivateKeyFromFile('.\domain.key');
          end
          else
          begin
               FTransport.UseSSL:=false;
               FTransport.Port:=80;
          end;
          FServer.Active:=true;
          btnListen.Caption:='Dont listen';
     end;
end;

Really simple code. Even the SSL part is extremely simple to setup in code.

Ok… there is a little bit more code to write. The following could have been set in properties on a WebSocket component put on the main form, but in this case I show it in code:

constructor TForm6.Create(Owner:TComponent);
begin
     inherited Create(Owner);
     FServer:=TkbmMWServer.Create(nil);
     FTransport:=TkbmMWWebSocketServerTransport.Create(nil);
     FTransport.Server:=FServer;
     FTransport.OnWebSocketData:=DoOnWebSocketData;
     FTransport.OnWebSocketOpen:=DoOnWebSocketOpen;
     FTransport.OnWebSocketClose:=DoOnWebSocketClose;
     FTransport.OnWebSocketPong:=DoOnWebSocketPong;
     FServer.AutoRegisterServices;
     Log.OutputToStrings(mLog.Lines);
end;

All that this code does is to create a kbmMW Server instance (TkbmMWServer) and a TkbmMWebSocketServerTransport instance. Those are components and could have been put on the form instead of instantiated in code.

I add some event handlers for when the WebSocket protocol receives data, when a client connects and disconnects and when a ping response is returned by a client as a pong.

Further I link the transport to the server component, and ask kbmMW to do its magic about discovering services that can be made available.

Finally I instruct the kbmMW logger to output all its log live to the TMemo.

And similarly there needs to be a little bit clean up code in the forms destructor:

destructor TForm6.Destroy;
begin
     FServer.Active:=false;
     FTransport.Free;
     FServer.Free;
     inherited Destroy;
end;

The following are event handlers for the WebSocket communication:

procedure TForm6.DoOnWebSocketData(const AConnection:IkbmMWWebSocketConnection; const AData:TValue);
var
   s:string;
begin
     s:=AData.ToString;
     Log.Info('Got data:'+s);
     AConnection.BroadcastText(s);
end;

procedure TForm6.DoOnWebSocketPong(const AConnection:IkbmMWWebSocketConnection; const AData:TValue);
begin
     Log.Info('Got pong from:'+
              AConnection.GetPeerAddr+':'+inttostr(AConnection.GetPeerPort)+
              ' : '+AData.ToString);
end;

procedure TForm6.DoOnWebSocketOpen(const AConnection:IkbmMWWebSocketConnection);
begin
     AConnection.TagString:='TAG1';
     Log.Info('Opening websocket connection: '+
              AConnection.GetPeerAddr+':'+inttostr(AConnection.GetPeerPort)+
              ' now tagged as '+AConnection.TagString);
end;

procedure TForm6.DoOnWebSocketClose(const AConnection:IkbmMWWebSocketConnection; const ACode:word; const AReason:string);
begin
     Log.Info('Closing websocket connection: '+
              AConnection.GetPeerAddr+':'+inttostr(AConnection.GetPeerPort)+
              ' Code:'+inttostr(ACode)+' Reason:'+AReason);
end;

The only one really required of these event handlers is the OnWebSocketData. In our case it simply broadcasts the data back to all WebSocket connected clients.

We still need a little bit more code.. namely something to verify and handle the request to upgrade from the HTTP protocol to the WebSocket protocol.

This is done in the HTTP service that is the 2nd Delphi unit added to the project.

The service was created by using the kbmMW service wizard:

The kbmMW Service wizard entry point in RAD Studio

After starting it, choose HTTP Smart Service

And then simply page right until you reach the end of the wizard and let the wizard generate the unit.

That results in this:

unit uHTTPService;

// =========================================================================
// kbmMW - An advanced and extendable middleware framework.
// by Components4Developers (http://www.components4developers.com)
//
// Service generated by kbmMW service wizard.
//
// INSTRUCTIONS FOR REGISTRATION/USAGE
// -----------------------------------
// Please update the uses clause of the datamodule/form the TkbmMWServer is placed on by adding services unit name 
// to it. Eg.
// 
//     uses ...,kbmMWServer,YourServiceUnitName;
// 
// Somewhere in your application, make sure to register the serviceclass to the TkbmMWServer instance.
// This can be done by registering the traditional way, or by using auto registration.
// 
// Traditional registration
// ------------------------
// var
//    sd:TkbmMWCustomServiceDefinition;
// ..
//    sd:=kbmMWServer1.RegisterService(yourserviceclassname,false);
// 
// Set the last parameter to true if this is the default service.
// 
// 
// Auto registration
// -----------------
// Make sure that your service class is tagged with the [kbmMW_Service] attribute.
// Then auto register all tagged services:
// ..
//    kbmMWServer1.AutoRegisterServices;
// 
// -----------------------------------------------
// 
// SPECIFIC HTTP SERVICE REGISTRATION INSTRUCTIONS
// -----------------------------------------------
// Cast the returned service definition object (sd) to a TkbmMWHTTPServiceDefinition. eg:
// 
// var
//    httpsd:TkbmMWHTTPServiceDefinition;
// ..
//    httpsd:=TkbmMWHTTPServiceDefinition(sd)
//    httpsd.RootPath[mwhfcHTML]:='.';
//    httpsd.RootPath[mwhfcImage]:='.';
//    httpsd.RootPath[mwhfcJavascript]:='.';
//    httpsd.RootPath[mwhfcStyleSheet]:='.';
//    httpsd.RootPath[mwhfcOther]:='.';
// -----------------------------------------------



{$I kbmMW.inc}

interface

uses
  SysUtils,
{$ifdef LEVEL6}
  Variants,
{$else}
  Forms,
{$endif}
  Classes,
  kbmMWSecurity,
  kbmMWServer,
  kbmMWServiceUtils,
  kbmMWHTTPStdTransStream,
  kbmMWGlobal,
  kbmMWCustomHTTPSmartService
   ,kbmMWRTTI
   ,kbmMWSmartServiceUtils;

type

  [kbmMW_Service('flags:[listed]')]
  [kbmMW_Rest('path:/')]
  // Access to the service can be limited using the [kbmMW_Auth..] attribute.
  // [kbmMW_Auth('role:[SomeRole,SomeOtherRole], grant:true')]

  TWebSocketService = class(TkbmMWCustomHTTPSmartService)
    function kbmMWCustomHTTPSmartServiceUpgrade(Sender: TObject;
      const ARequestHelper, AResponseHelper: TkbmMWHTTPTransportStreamHelper;
      const ARequestedProtocols: string;
      var AAcceptedProtocol: string): Boolean;
  private
     { Private declarations }
  protected
     { Protected declarations }
  public
     { Public declarations }
     // HelloWorld function callable from both a regular client,
     // due to the optional [kbmMW_Method] attribute,
     // and from a REST client due to the optional [kbmMW_Rest] attribute.
     // The access path to the function from a REST client (like a browser)+
     // is in this case relative to the services path.
     // In this example: http://...//helloworld
     // Access to the function can be limited using the [kbmMW_Auth..] attribute.
     // [kbmMW_Auth('role:[SomeRole,SomeOtherRole], grant:true')]
     [kbmMW_Rest('method:get, path:helloworld')]
     [kbmMW_Method]
     function HelloWorld:string;
  end;

implementation

uses kbmMWExceptions, uMain;

{$R *.dfm}


// Service definitions.
//---------------------

function TWebSocketService.HelloWorld:string;
begin
     Result:='Hello world';
end;

function TWebSocketService.kbmMWCustomHTTPSmartServiceUpgrade(Sender: TObject;
  const ARequestHelper, AResponseHelper: TkbmMWHTTPTransportStreamHelper;
  const ARequestedProtocols: string; var AAcceptedProtocol: string): Boolean;
begin
     AAcceptedProtocol:='websocket';
     Result:=true;
end;

initialization
  TkbmMWRTTI.EnableRTTI(TWebSocketService);
end.

It contains a lengthy comment at start, explaining a little bit about what the service is supposed to do and how to register it with kbmMW.

But the interesting part is the function HelloWorld and the OnUpgrade event handler.

The function HelloWorld is just included as standard to show how to create a function that can be called from a browser, and which returns some data or HTML or other stuff to be presented in the client. It is possible to tell which mimetype is returned, so the browser will display the data correctly. In this case we simply return html/text since no specific mimetype is given.

You call this function from the browser by requesting an URL like this (provided SSL is not enabled, otherwise replace http with https):

http://localhost/helloworld

The browser will then display the text returned… Hello world.

You could return files and many other things here, but you can also just leave that up to kbmMW, who will try to locate a file to send to the browser in case no services are implemented to handle the request (URL path).

The even handler OnUpgrade is called the moment a client asks for upgrading the connection to some other sort of connection. The client will provide a string over protocols that it will accept. In this case you will receive this list as a comma separated string.

The event handlers job, is then to return back to the client, one of the protocols that the server will accept. In the case of WebSockets, it is the websocket protocol written exactly like that in lowercase.

As long as the client requested the ability to upgrade to WebSocket, you will now have established a WebSocket connection between the client and the kbmMW server.

If the client asked for something else and did not include websocket as one of its protocol options, the client will disconnect with an error.

Now that is _all_ there is need as far as server side code. Let us take a look at the client side.

Client side code

The client side (the browser) will be fed with the needed HTML and Javascript from the server to allow implementation of a chat client that is able to open a WebSocket connection to the server.

This is done, in this sample, by providing the index.html file to the client when it connects at first.
The index.html file is relatively simple. It contains a little bit of HTML, primarily some divisions for text entry and the chat log, and then it contains a little bit of styling (CSS) and finally it contains a little bit of Javascript code to handle the WebSocket communication.

<!DOCTYPE html>
<html lang="en">

<head>
    <title>WebSocket Demo</title>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <style>
        #chatFormContainer {
            text-align: center;
            position: fixed;
            bottom: 5px;
            left: 5px;
            right: 5px;
        }

        #chatMessage {
            display: inline-block;
            float: left;
            width: 90%;
            height: 20px;
            padding: 10px;
        }
    </style>
</head>

<body>
    <div id="chatLog">

    </div>
    <div id="chatFormContainer">
        <form id="chatForm">
            <input id="chatMessage" placeholder="Type  message and press enter..."
                   size="1">
        </form>
    </div>
    <script>
        var username = `user_${getRandomNumber()}`
        var channelId = 1;
        var socket = new WebSocket(location.origin.replace(/^http/, 'ws') + 
                                   '/v3/${channelId}?notify_self');

        var chatLog = document.getElementById('chatLog');
        var chatForm = document.getElementById('chatForm');
        chatForm.addEventListener("submit", sendMessage);

        socket.onopen = function() {
            console.log(`Websocket connected`);
            socket.send(JSON.stringify({
                event: 'new_joining',
                sender: username,
            }));
        }
        socket.onmessage = function(message) {
            var payload = JSON.parse(message.data);
            console.log(payload);

            if (payload.sender == username) {
                payload.sender = "You";
            }

            if (payload.event == "new_message") {
                //Handle new message
                chatLog.insertAdjacentHTML('afterend', 
                                 `<div> ${payload.sender}: ${payload.text} <div>`);
            }
            else if (payload.event == 'new_joining') {
                //Handle new joining
                chatLog.insertAdjacentHTML('afterend', 
                                 `<div> ${payload.sender} joined the chat<div>`);
            }
        }

        function getRandomNumber() {
            return Math.floor(Math.random() * 1000);
        }

        function sendMessage(e) {
            e.preventDefault();

            var message = document.getElementById('chatMessage').value;

            socket.send(JSON.stringify({
                event: 'new_message',
                sender: username,
                text: message
            }));
        }
    </script>
</body>
</html>

First the HTML is rendered (the two div’s one of them holding a simple form). Secondly the script is run.
The script attempts to establish a WebSocket connection to the connected server, sets up some event handlers for data and states coming from the WebSocket, and sets up a couple of event handlers on the form’s input component so it will send the input via the WebSocket connection the moment Enter is pressed (submit).

The result: