2016/12/19

Websocket in Scala Play

Websocket 就是在網頁 http protocol 之下,再增加的一層封裝協定,目的是讓 client 建立長時間的 socket 連線,在不重新建立連線的狀況下,我們可以傳送更多即時的資訊。

根據 WebSockets 的官方文件說明,scala play 2.5版已經調整為使用 Akka Streams 及 actors 來處理 WebSockets。

以最基本的 String 資料格式為例,跟一般的 http Action 一樣,websocket 也是以 Controller 為入口點。

WebSocketController 有兩個部分,index 是回傳一個簡單的 websocket 網頁 client,而 socket 就是主要處理 websocket 的部分。

// WebSocketController.scala
import play.api.mvc._
import play.api.libs.streams._

class WebSocketController @Inject()(implicit system: ActorSystem, materializer: Materializer) extends Controller {

  def index = Action { implicit request =>
    Ok(views.html.ws())
  }

  def socket = WebSocket.accept[String, String] { request =>
    ActorFlow.actorRef(out => MyWebSocketActor.props(out))
  }
}

ActorFlow.actorRef(...) 可以被替代為 Akka Streams Flow[In, Out, _],但還是用 actors 撰寫會比較直接。

在 WebSocketController.scala 的下面,我們直接將處理 websocket 的 MyWebSocketActor 寫在下面,收到字串就會直接回傳 echo String: 加上原本收到的字串。

import akka.actor._

object MyWebSocketActor {
  def props(out: ActorRef) = Props(new MyWebSocketActor(out))
}

class MyWebSocketActor(out: ActorRef) extends Actor {
  def receive = {
    case msg: String =>
      Logger.debug(s"echo String: ${msg}")
      out ! s"echo String: ${msg}"
  }
}

在 routes 加上兩個 service uri

GET         /ws                                         controllers.WebSocketController.socket
GET         /wstest                                     controllers.WebSocketController.index

如果不要自己撰寫 websocket client 網頁,可以直接用 Websocket Echo Test 測試,把 Location: wss://echo.websocket.org 改為 ws://localhost:9000/ws,Use secure WebSocket (TLS) 取消勾選。點 Connect 以後,就可以對 ws server 發送文字訊息。

以下是自己撰寫 websocket 網頁的 sample: ws.scala.html

<!DOCTYPE html>
<meta charset="utf-8" />
<title>WebSocket Test</title>
<script language="javascript" type="text/javascript">
    //var wsUri = "ws://localhost:9000/ws";
    //var output;

    function init()
    {
        //output = document.getElementById("output");
        //testWebSocket();
    }

    function testWebSocket()
    {
        var wsUri= document.getElementById('wsUri').value;
        websocket = new WebSocket(wsUri);
        //websocket.binaryType = 'arraybuffer';

        websocket.onopen = function(evt) { onOpen(evt) };
        websocket.onclose = function(evt) { onClose(evt) };
        websocket.onmessage = function(evt) { onMessage(evt) };
        websocket.onerror = function(evt) { onError(evt) };
    }

    function closeWebSocket() {
        websocket.close();
    }

    function onOpen(evt)
    {
        writeToScreen("CONNECTED");
        //doSend("WebSocket rocks");
    }

    function onClose(evt)
    {
        writeToScreen("DISCONNECTED");
    }

    function onMessage(evt)
    {
        console.log(evt);

        if ( evt.data instanceof ArrayBuffer ) {
            console.log( evt.data );
        } else {
            writeToScreen('<span style="color: blue;">RESPONSE: ' + evt.data+'</span>');
        }
        //websocket.close();
    }

    function onError(evt)
    {
        writeToScreen('<span style="color: red;">ERROR:</span> ' + evt.data);
    }

//    function doSend(message)
//    {
//        writeToScreen("SENT: " + message);
//        websocket.send(message);
//    }

    function doSendInput()
    {
        var message= document.getElementById('sendMessage').value;

        writeToScreen("SENT: " + message);
        websocket.send( message );
        //websocket.send( JSON.stringify(JSON.parse(message)) );
        //var byteArray = new Uint8Array(message);
        //websocket.send( byteArray.buffer );
    }

    function writeToScreen(message)
    {
        var pre = document.createElement("p");
        pre.style.wordWrap = "break-word";
        pre.innerHTML = message;

        output = document.getElementById("output");
        output.appendChild(pre);
    }

    function clearScreen()
    {
        var nodes = document.getElementById("output");
        while (nodes.hasChildNodes()) {
            nodes.removeChild(nodes.childNodes[0]);
        }
    }

    window.addEventListener("load", init, false);

    </script>

<h2>WebSocket Test</h2>

<input id="wsUri" size="35" value="ws://localhost:9000/ws"></input>

<button id="close" onclick="javascipt:testWebSocket()">connect</button>
<button id="close" onclick="javascipt:closeWebSocket()">close</button>

<br/><br/>

<input id="sendMessage" size="35" value="測試 WebSocket!!!"></input>
<button id="send" onclick="javascipt:doSendInput()">Send</button>
<button id="clear" onclick="javascipt:clearScreen()">clear</button>
<div id="output"></div>

啟動 server 瀏覽網頁 http://localhost:9000/wstest 就可以測試 websocket

如果要將資料的格式換成 json,就稍微調整一下程式。

//WebSocketController.scala 把 String 換成 JsValue

  def socketJs = WebSocket.accept[JsValue, JsValue] { request =>
    ActorFlow.actorRef(out => MyWebSocketActor.props(out))
  }

// 接收的資料改為 JsValue,並直接回傳原始的 json
class MyWebSocketActor(out: ActorRef) extends Actor {
  def receive = {
    case msg: JsValue =>
      Logger.debug(s"echo JsValue: ${msg}")
      out ! msg
  }
}

routes 修改為 socketJs method

GET         /ws                                         controllers.WebSocketController.socketJs
GET         /wstest                                     controllers.WebSocketController.index

ws.scala.html 中發送資料的部分,以 JSON.parse 及 JSON.stringify 確認發送的資料是 json。

    function doSendInput()
    {
        var message= document.getElementById('sendMessage').value;

        writeToScreen("SENT: " + message);
        //websocket.send( message );
        websocket.send( JSON.stringify(JSON.parse(message)) );
    }

測試的字串要改為 json

References

Binary websockets with Play 2.0 and Scala (and a bit op JavaCV/OpenCV)

Handling data streams reactively

Akka Streams integration in Play Framework 2.5

2016/12/12

How to use XML in scala

scala 內建了 scala.xml 可以處理 XML,以下我們簡單測試了一下 xml 的功能,目前看起來,標準函式庫的功能就足夠應用的需求了。

產生及使用 scala.xml

scala 可以直接偵測 < 這個符號,並以 xml 的方式處理,在程式中,直接把 xml指定給變數,就會直接產生 xml 物件。

scala> val xml = <store><item type="book">Novel</item><item type="pen">pencil</item></store>
xml: scala.xml.Elem = <store><item type="book">Novel</item><item type="pen">pencil</item></store>

產生 xml 物件後,可以用 text method,將所有欄位的 value 全部印出來

scala> xml.text
res0: String = Novelpencil

用 XPath 的語法,可以取得子節點的內容

scala> xml\"item"
res1: scala.xml.NodeSeq = NodeSeq(<item type="book">Novel</item>, <item type="pen">pencil</item>)

scala> (xml\"item").map(_.text).mkString(" ")
res2: String = Novel pencil

用 "@type" selector,可以取得所有 type 屬性的值

scala> (xml\"item").map(_\"@type")
res3: scala.collection.immutable.Seq[scala.xml.NodeSeq] = List(book, pen)

"@type" 只能取得目前這個節點的 children,如果要將所有出現同樣屬性的節點都取出來,就要用 "\" selector

scala> znodes\"z"
res5: scala.xml.NodeSeq = NodeSeq(<z x="1"/>)

scala> znodes\\"z"
res6: scala.xml.NodeSeq = NodeSeq(<z x="1"/>, <z x="2"/>, <z x="3"/>, <z x="4"/>)

跟剛剛一樣的方法,就可以找到屬性 x 的所有 values

scala> (znodes\\"z").map(_\"@x")
res12: scala.collection.immutable.Seq[scala.xml.NodeSeq] = List(1, 2, 3, 4)

如果是一個 xml string,可以用 scala.xml.XML.loadString 將 string 轉換為 xml。

scala> val xmlstring = """<store><item type="book">Novel</item><item type="pen">pencil</item></store>"""
xmlstring: String = <store><item type="book">Novel</item><item type="pen">pencil</item></store>

scala> val xmlfromstring = scala.xml.XML.loadString(xmlstring)
xml: scala.xml.Elem = <store><item type="book">Novel</item><item type="pen">pencil</item></store>

scala> xml == xmlfromstring
res13: Boolean = true

manual Serializing and Deserializing xml

首先定義 Stock class,在 toXml method 中,可以直接用 {symbol} 將 Stock 物件中的資料,產生到 xml 中,而 fromXml 就是用剛剛的 text 的方式取得 xml 裡面的資料放入 Stock 物件中。

case class Stock(var symbol: String, var businessName: String, var price: Double) {

  // convert Stock fields to XML
  def toXml = {
    <stock>
      <symbol>{symbol}</symbol>
      <businessName>{businessName}</businessName>
      <price>{price}</price>
    </stock>
  }

  override def toString =
    s"symbol: $symbol, businessName: $businessName, price: $price"

}

object Stock {

  // convert XML to a Stock
  def fromXml(node: scala.xml.Node):Stock = {
    val symbol = (node \ "symbol").text
    val businessName = (node \ "businessName").text
    val price = (node \ "price").text.toDouble
    new Stock(symbol, businessName, price)
  }
}

用個簡單的 TestXml 測試 toXml 及 fromXml

object TestXml extends App {

  // convert a Stock to its XML representation
  val aapl = new Stock("AAPL", "Apple", 600d)
  println(aapl.toXml)

  // convert an XML representation to a Stock
  val googXml = <stock>
    <symbol>GOOG</symbol>
    <businessName>Google</businessName>
    <price>620.00</price>
  </stock>
  val goog = Stock.fromXml(googXml)
  println(goog)
}

執行結果如下

<stock>
      <symbol>AAPL</symbol>
      <businessName>Apple</businessName>
      <price>600.0</price>
    </stock>

symbol: GOOG, businessName: Google, price: 620.0

References

Basic XML processing with Scala

Practical Scala – processing XML

Serializing and deserializing XML in Scala

Generating dynamic XML from Scala source code (like a template)

2016/12/05

Play JSON library

Scala Play 支援自己實作的 play.api.libs.json JSON Lirary

類別

  • JsValue: trait 代表任意一種 JSON value,以下這些是 extend JsValue 的 case classes

  1. JsString
  2. JsNumber
  3. JsBoolean
  4. JsObject
  5. JsArray
  6. JsNull

  • Json 提供 utilties,轉換 JsValue

  • JsPath 代表 JsValue 資料結構中的 path,類似 XML 的 XPath,可使用 pattern 在 JsValue 中進行 data traversal

Converting to a JsValue

  • 以 json String 直接 parsing 並轉換
import play.api.libs.json._

val json: JsValue = Json.parse("""
{
  "name" : "Watership Down",
  "location" : {
    "lat" : 51.235685,
    "long" : -1.309197
  },
  "residents" : [ {
    "name" : "Fiver",
    "age" : 4,
    "role" : null
  }, {
    "name" : "Bigwig",
    "age" : 6,
    "role" : "Owsla"
  } ]
}
""")

直接將上面的 code ,在 activator console 中進行測試,就可以得到測試結果

scala> json
res0: play.api.libs.json.JsValue = {"name":"Watership Down","location":{"lat":51.235685,"long":-1.309197},"residents":[{"name":"Fiver","age":4,"role":null},{"name":"Bigwig","age":6,"role":"Owsla"}]}
  • 以 class 的方式建立 JsValue
import play.api.libs.json._

val json: JsValue = JsObject(Seq(
  "name" -> JsString("Watership Down"),
  "location" -> JsObject(Seq("lat" -> JsNumber(51.235685), "long" -> JsNumber(-1.309197))),
  "residents" -> JsArray(Seq(
    JsObject(Seq(
      "name" -> JsString("Fiver"),
      "age" -> JsNumber(4),
      "role" -> JsNull
    )),
    JsObject(Seq(
      "name" -> JsString("Bigwig"),
      "age" -> JsNumber(6),
      "role" -> JsString("Owsla")
    ))
  ))
))

使用 Writes converters

scala converting to a JsValue 是以 Json.toJsonT(implicit writes: Writes[T]) 進行轉換的,這需要 Writes 物件的幫忙。Play JSON API 提供了一個 implicit Writes 物件,可以轉換 Int, Double, String, Boolean ...,同時也支援 collections

import play.api.libs.json._

// basic types
val jsonString = Json.toJson("Fiver")
val jsonNumber = Json.toJson(4)
val jsonBoolean = Json.toJson(false)

// collections of basic types
val jsonArrayOfInts = Json.toJson(Seq(1, 2, 3, 4))
val jsonArrayOfStrings = Json.toJson(List("Fiver", "Bigwig"))

如果要轉換自己的資料物件類別,就須要提供一個 implicit Writes converter

case class Location(lat: Double, long: Double)
case class Resident(name: String, age: Int, role: Option[String])
case class Place(name: String, location: Location, residents: Seq[Resident])
import play.api.libs.json._

implicit val locationWrites = new Writes[Location] {
  def writes(location: Location) = Json.obj(
    "lat" -> location.lat,
    "long" -> location.long
  )
}

implicit val residentWrites = new Writes[Resident] {
  def writes(resident: Resident) = Json.obj(
    "name" -> resident.name,
    "age" -> resident.age,
    "role" -> resident.role
  )
}

implicit val placeWrites = new Writes[Place] {
  def writes(place: Place) = Json.obj(
    "name" -> place.name,
    "location" -> place.location,
    "residents" -> place.residents)
}

val place = Place(
  "Watership Down",
  Location(51.235685, -1.309197),
  Seq(
    Resident("Fiver", 4, None),
    Resident("Bigwig", 6, Some("Owsla"))
  )
)

val json = Json.toJson(place)

也可以使用 combinator pattern

import play.api.libs.json._
import play.api.libs.functional.syntax._

implicit val locationWrites: Writes[Location] = (
  (JsPath \ "lat").write[Double] and
  (JsPath \ "long").write[Double]
)(unlift(Location.unapply))

implicit val residentWrites: Writes[Resident] = (
  (JsPath \ "name").write[String] and
  (JsPath \ "age").write[Int] and
  (JsPath \ "role").writeNullable[String]
)(unlift(Resident.unapply))

implicit val placeWrites: Writes[Place] = (
  (JsPath \ "name").write[String] and
  (JsPath \ "location").write[Location] and
  (JsPath \ "residents").write[Seq[Resident]]
)(unlift(Place.unapply))

Traversing a JsValue structure

可以 traverse JsValue 並取得特定的資料欄位,這就類似 Scala XML processing

  • Simple Path \
val lat = (json \ "location" \ "lat").get
// returns JsNumber(51.235685)
  • Recursive path \
val names = json \\ "name"
// returns Seq(JsString("Watership Down"), JsString("Fiver"), JsString("Bigwig"))
  • Index lookup (針對 JsArrays)
val bigwig = (json \ "residents")(1)
// returns {"name":"Bigwig","age":6,"role":"Owsla"}

Converting from a JsValue

  • 使用 String utilities
val minifiedString: String = Json.stringify(json)

適合閱讀的格式

val readableString: String = Json.prettyPrint(json)
  • 使用 JsValue.as/asOpt

將 JsValue 轉換為其他資料類別的方式,最簡單的方式就是用 JsValue.asT: T ,但這個方式需要用到一個 implicit converter: Reads[T]

val name = (json \ "name").as[String]
// "Watership Down"

val names = (json \\ "name").map(_.as[String])
// Seq("Watership Down", "Fiver", "Bigwig")

但是 as 可能會產生 JsResultException,所以比較好的方式,應該是改用 JsValue.asOptT: Option[T]

val nameOption = (json \ "name").asOpt[String]
// Some("Watership Down")

val bogusOption = (json \ "bogus").asOpt[String]
// None
  • 使用 validation

會產生兩種 JsResult

  1. JsSuccess: validation/convertion 成功
  2. JsError: validation/conversion 失敗
val json = { ... }

val nameResult: JsResult[String] = (json \ "name").validate[String]

// Pattern matching
nameResult match {
  case s: JsSuccess[String] => println("Name: " + s.get)
  case e: JsError => println("Errors: " + JsError.toJson(e).toString())
}

// Fallback value
val nameOrFallback = nameResult.getOrElse("Undefined")

// map
val nameUpperResult: JsResult[String] = nameResult.map(_.toUpperCase())

// fold
val nameOption: Option[String] = nameResult.fold(
  invalid = {
    fieldErrors => fieldErrors.foreach(x => {
      println("field: " + x._1 + ", errors: " + x._2)
    })
    None
  },
  valid = {
    name => Some(name)
  }
)
  • JsValue 轉換成 data model
case class Location(lat: Double, long: Double)
case class Resident(name: String, age: Int, role: Option[String])
case class Place(name: String, location: Location, residents: Seq[Resident])
import play.api.libs.json._
import play.api.libs.functional.syntax._

implicit val locationReads: Reads[Location] = (
  (JsPath \ "lat").read[Double] and
  (JsPath \ "long").read[Double]
)(Location.apply _)

implicit val residentReads: Reads[Resident] = (
  (JsPath \ "name").read[String] and
  (JsPath \ "age").read[Int] and
  (JsPath \ "role").readNullable[String]
)(Resident.apply _)

implicit val placeReads: Reads[Place] = (
  (JsPath \ "name").read[String] and
  (JsPath \ "location").read[Location] and
  (JsPath \ "residents").read[Seq[Resident]]
)(Place.apply _)


val json = { ... }

val placeResult: JsResult[Place] = json.validate[Place]
// JsSuccess(Place(...),)

val residentResult: JsResult[Resident] = (json \ "residents")(1).validate[Resident]
// JsSuccess(Resident(Bigwig,6,Some(Owsla)),)

Reference

Scala Json in Play

JSON_basics