2021/10/18

Piping-server

piping-server 可透過 http/https 在 devices 之間傳遞任意資料,可單獨傳送文字或是檔案。也可以使用串流的方式,無限制一直傳送資料。透過這個機制,能夠進一步實現 Text stream chat, screen share, dawing share, E2E encryption file transfer, Web ssh, WebVNC。經過實驗,可持續傳送 64 天的資料,大約傳送了 1PB。

public servers

如果不自行安裝,目前有這些 public servers 可直接測試使用

測試

傳送文字

# Send
echo 'hello, world' | curl -T - https://ppng.io/hello

# Get
curl https://ppng.io/hello > hello.txt

server url (/hello) 路徑可任意指定,將資料透過 PUT/POST,然後用 GET 取得資料,傳送或接收任一方都可以先發起。

傳送檔案

# send
curl -T file https://ppng.io/file
# send with linux pipe
cat file | curl -T - https://ppng.io/file

# receive
curl https://ppng.io/file > file

傳送資料夾

# send with tar
tar zfcp - ~/mydirectory | curl -T - https://ppng.io/dir

# receive
curl https://ppng.io/dir > dir.tar

##########
# send with zip
zip -q -r - ~/mydirctory | curl -T - https://ppng.io/dir

# receive
curl https://ppng.io/dir > dir.zip

傳送檔案並加密

# send
cat file | openssl aes-256-cbc | curl -T - https://ppng.io/encfile

# receive
curl https://ppng.io/encfile | openssl aes-256-cbc -d > file2

指定多個接收端

注意發送端會等待兩個接收端都連上,才會開始進行資料傳輸,否則,當第一個接收端連上時,還是會在等待的狀態

#send
seq 10 | curl -T - https://ppng.io/seq?n=2

# 在兩個 terminal receive
curl https://ppng.io/seq?n=2
curl https://ppng.io/seq?n=2

portable execution

piping-server-pkg/releases 可下載平台的 binary package 執行檔,然後就能直接使用

以下是用 piping-server-linux 搭配 Let's Encrypt 的 SSL 憑證的啟動方式

./piping-server-linux --http-port 9000 --enable-https --https-port 9001 --key-path /etc/letsencrypt/live/testserver.com.tw/privkey.pem --crt-path /etc/letsencrypt/live/testserver.com.tw/fullchain.pem

線上直接查詢 server 的版本及 help

curl https://ppng.io/help

curl https://ppng.io/version

applications

screen share

draw web

References

Piping Server:實現設備間通過純 HTTP 無限傳輸數據

Transfer Files Between Any Devices Using Piping Server

piping-server-streaming-upload-htmls

2021/10/04

RTCDataChannel Sample

Transmit text

Transmit text

單獨一個網頁,自己產生 local 及 remote peer connection,然後兩個產生 data channel 直接互相連接

    <div id="buttons">
        <button id="startButton">Start</button>
        <button id="sendButton" disabled>Send</button>
        <button id="closeButton" disabled>Stop</button>
    </div>

    <div id="sendReceive">
        <div id="send">
            <h2>Send</h2>
            <textarea id="dataChannelSend" disabled
                      placeholder="Press Start, enter some text, then press Send."></textarea>
        </div>
        <div id="receive">
            <h2>Receive</h2>
            <textarea id="dataChannelReceive" disabled></textarea>
        </div>
    </div>
/*
 *  Copyright (c) 2015 The WebRTC project authors. All Rights Reserved.
 *
 *  Use of this source code is governed by a BSD-style license
 *  that can be found in the LICENSE file in the root of the source
 *  tree.
 */

'use strict';

let localConnection;
let remoteConnection;
let sendChannel;
let receiveChannel;
const dataChannelSend = document.querySelector('textarea#dataChannelSend');
const dataChannelReceive = document.querySelector('textarea#dataChannelReceive');
const startButton = document.querySelector('button#startButton');
const sendButton = document.querySelector('button#sendButton');
const closeButton = document.querySelector('button#closeButton');

startButton.onclick = createConnection;
sendButton.onclick = sendData;
closeButton.onclick = closeDataChannels;

function enableStartButton() {
  startButton.disabled = false;
}

function disableSendButton() {
  sendButton.disabled = true;
}

function createConnection() {
  dataChannelSend.placeholder = '';
  const servers = null;

  // 產生 local peer connection
  window.localConnection = localConnection = new RTCPeerConnection(servers);
  console.log('Created local peer connection object localConnection');

  // 由 localConnection 產生 DataChannel
  sendChannel = localConnection.createDataChannel('sendDataChannel');
  console.log('Created send data channel');

  // 2個 callback function: onIceCandidate, onSendChannelStateChange
  localConnection.onicecandidate = e => {
    onIceCandidate(localConnection, e);
  };
  sendChannel.onopen = onSendChannelStateChange;
  sendChannel.onclose = onSendChannelStateChange;


  //////////////
  // remote peer connection
  window.remoteConnection = remoteConnection = new RTCPeerConnection(servers);
  console.log('Created remote peer connection object remoteConnection');

  // 2 個 callback function: onIceCandidate, receiveChannelCallback
  remoteConnection.onicecandidate = e => {
    onIceCandidate(remoteConnection, e);
  };
  remoteConnection.ondatachannel = receiveChannelCallback;

  ///////////
  // 由 localConnection 產生 offer -> setLocalDescription
  localConnection.createOffer().then(
      gotDescription1,
      onCreateSessionDescriptionError
  );
  startButton.disabled = true;
  closeButton.disabled = false;
}

function onCreateSessionDescriptionError(error) {
  console.log('Failed to create session description: ' + error.toString());
}

function sendData() {
  // 透過 send data channel 發送訊息
  const data = dataChannelSend.value;
  sendChannel.send(data);
  console.log('Sent Data: ' + data);
}

function closeDataChannels() {
  console.log('Closing data channels');
  sendChannel.close();
  console.log('Closed data channel with label: ' + sendChannel.label);
  receiveChannel.close();
  console.log('Closed data channel with label: ' + receiveChannel.label);
  localConnection.close();
  remoteConnection.close();
  localConnection = null;
  remoteConnection = null;
  console.log('Closed peer connections');
  startButton.disabled = false;
  sendButton.disabled = true;
  closeButton.disabled = true;
  dataChannelSend.value = '';
  dataChannelReceive.value = '';
  dataChannelSend.disabled = true;
  disableSendButton();
  enableStartButton();
}

function gotDescription1(desc) {
  // 設定 local connection 的 local description
  localConnection.setLocalDescription(desc);
  console.log(`Offer from localConnection\n${desc.sdp}`);

  // 將 local description 直接設定給 remote connection 的 remote description
  remoteConnection.setRemoteDescription(desc);
  // 由 remote connection 產生 answer, 呼叫 gotDescription2
  remoteConnection.createAnswer().then(
      gotDescription2,
      onCreateSessionDescriptionError
  );
}

function gotDescription2(desc) {
  // 設定 remote connection 的 local description
  remoteConnection.setLocalDescription(desc);
  console.log(`Answer from remoteConnection\n${desc.sdp}`);

  // 設定 local connection 的 remote description
  localConnection.setRemoteDescription(desc);
}


//////////////////
function getOtherPc(pc) {
  return (pc === localConnection) ? remoteConnection : localConnection;
}

function getName(pc) {
  return (pc === localConnection) ? 'localPeerConnection' : 'remotePeerConnection';
}

function onIceCandidate(pc, event) {
  // 取得 local connection 的 ice candidate, 指定給 remote connection 的 addIceCandidate
  // 取得 remote connection 的 ice candidate, 指定給 local connection 的 addIceCandidate

  getOtherPc(pc)
      .addIceCandidate(event.candidate)
      .then(
          onAddIceCandidateSuccess,
          onAddIceCandidateError
      );
  console.log(`${getName(pc)} ICE candidate: ${event.candidate ? event.candidate.candidate : '(null)'}`);
}

function onAddIceCandidateSuccess() {
  console.log('AddIceCandidate success.');
}

function onAddIceCandidateError(error) {
  console.log(`Failed to add Ice Candidate: ${error.toString()}`);
}

function receiveChannelCallback(event) {
  // 產生 receive data channel
  console.log('Receive Channel Callback');
  receiveChannel = event.channel;
  receiveChannel.onmessage = onReceiveMessageCallback;
  receiveChannel.onopen = onReceiveChannelStateChange;
  receiveChannel.onclose = onReceiveChannelStateChange;
}

function onReceiveMessageCallback(event) {
  // 收到訊息
  console.log('Received Message');
  dataChannelReceive.value = event.data;
}

function onSendChannelStateChange() {
  const readyState = sendChannel.readyState;
  console.log('Send channel state is: ' + readyState);
  if (readyState === 'open') {
    // 當 local connection 的 data channel 為 open 時, 改變 UI 元件的狀態
    dataChannelSend.disabled = false;
    dataChannelSend.focus();
    sendButton.disabled = false;
    closeButton.disabled = false;
  } else {
    dataChannelSend.disabled = true;
    sendButton.disabled = true;
    closeButton.disabled = true;
  }
}

function onReceiveChannelStateChange() {
  //
  const readyState = receiveChannel.readyState;
  console.log(`Receive channel state is: ${readyState}`);
}

Transfer a file

    <section>
      <div >
        <form id="fileInfo">
          <input type="file" id="fileInput" name="files"/>
        </form>
        <button disabled id="sendFile">Send</button>
        <button disabled id="abortButton">Abort</button>
      </div>

      <div class="progress">
        <div class="label">Send progress: </div>
        <progress id="sendProgress" max="0" value="0"></progress>
      </div>

      <div class="progress">
        <div class="label">Receive progress: </div>
        <progress id="receiveProgress" max="0" value="0"></progress>
      </div>

      <div id="bitrate"></div>
      <a id="download"></a>
      <span id="status"></span>

    </section>
/* eslint no-unused-expressions: 0 */
/*
 *  Copyright (c) 2015 The WebRTC project authors. All Rights Reserved.
 *
 *  Use of this source code is governed by a BSD-style license
 *  that can be found in the LICENSE file in the root of the source
 *  tree.
 */
'use strict';

let localConnection;
let remoteConnection;
let sendChannel;
let receiveChannel;
let fileReader;
const bitrateDiv = document.querySelector('div#bitrate');
const fileInput = document.querySelector('input#fileInput');
const abortButton = document.querySelector('button#abortButton');
const downloadAnchor = document.querySelector('a#download');
const sendProgress = document.querySelector('progress#sendProgress');
const receiveProgress = document.querySelector('progress#receiveProgress');
const statusMessage = document.querySelector('span#status');
const sendFileButton = document.querySelector('button#sendFile');

let receiveBuffer = [];
let receivedSize = 0;

let bytesPrev = 0;
let timestampPrev = 0;
let timestampStart;
let statsInterval = null;
let bitrateMax = 0;

sendFileButton.addEventListener('click', () => createConnection());
fileInput.addEventListener('change', handleFileInputChange, false);
abortButton.addEventListener('click', () => {
  // 取消 fileReader
  if (fileReader && fileReader.readyState === 1) {
    console.log('Abort read!');
    fileReader.abort();
  }
});

async function handleFileInputChange() {
  // 選擇新的檔案
  const file = fileInput.files[0];
  if (!file) {
    console.log('No file chosen');
  } else {
    sendFileButton.disabled = false;
  }
}

async function createConnection() {
  // 發送檔案按鈕 -> 產生連線
  abortButton.disabled = false;
  sendFileButton.disabled = true;

  // local peer connection
  localConnection = new RTCPeerConnection();
  console.log('Created local peer connection object localConnection');

  // send data channel
  sendChannel = localConnection.createDataChannel('sendDataChannel');
  sendChannel.binaryType = 'arraybuffer';
  console.log('Created send data channel');

  // onSendChannelStateChange
  sendChannel.addEventListener('open', onSendChannelStateChange);
  sendChannel.addEventListener('close', onSendChannelStateChange);
  sendChannel.addEventListener('error', onError);

  // icecandidate
  localConnection.addEventListener('icecandidate', async event => {
    console.log('Local ICE candidate: ', event.candidate);
    // 直接將 local connection 的 icecandidate 加入 remote connection
    await remoteConnection.addIceCandidate(event.candidate);
  });


  //////////////
  // remote peer connection
  remoteConnection = new RTCPeerConnection();
  console.log('Created remote peer connection object remoteConnection');

  // icecandidate
  remoteConnection.addEventListener('icecandidate', async event => {
    console.log('Remote ICE candidate: ', event.candidate);
    // 直接將 remote connection 的 icecandidate 加入 local connection
    await localConnection.addIceCandidate(event.candidate);
  });
  remoteConnection.addEventListener('datachannel', receiveChannelCallback);


  // 由 local connection 產生 offer, 取得 sdp description
  try {
    const offer = await localConnection.createOffer();
    await gotLocalDescription(offer);
  } catch (e) {
    console.log('Failed to create session description: ', e);
  }

  fileInput.disabled = true;
}

function sendData() {
  const file = fileInput.files[0];
  console.log(`File is ${[file.name, file.size, file.type, file.lastModified].join(' ')}`);

  // Handle 0 size files.
  statusMessage.textContent = '';
  downloadAnchor.textContent = '';
  if (file.size === 0) {
    bitrateDiv.innerHTML = '';
    statusMessage.textContent = 'File is empty, please select a non-empty file';
    closeDataChannels();
    return;
  }
  sendProgress.max = file.size;
  receiveProgress.max = file.size;

  // chunkSize  16 kB = 16 * 1024
  const chunkSize = 16384;
  fileReader = new FileReader();
  let offset = 0;
  fileReader.addEventListener('error', error => console.error('Error reading file:', error));
  fileReader.addEventListener('abort', event => console.log('File reading aborted:', event));
  fileReader.addEventListener('load', e => {
    console.log('FileRead.onload ', e);

    // 透過 sendChannel 發送 fileReader 讀取到的 array buffer
    sendChannel.send(e.target.result);
    offset += e.target.result.byteLength;
    sendProgress.value = offset;
    if (offset < file.size) {
      // 當 offset 小於 file.size,就持續一直做 readSlice
      readSlice(offset);
    }
  });
  const readSlice = o => {
    console.log('readSlice ', o);
    // 以 file.slice 取得 file 的 某個區塊
    const slice = file.slice(offset, o + chunkSize);
    fileReader.readAsArrayBuffer(slice);
  };

  // 讀取 第 0 個 bytes, 每一次讀 chunkSize 16k bytes
  readSlice(0);
}

function closeDataChannels() {
  console.log('Closing data channels');
  sendChannel.close();
  console.log(`Closed data channel with label: ${sendChannel.label}`);
  sendChannel = null;
  if (receiveChannel) {
    receiveChannel.close();
    console.log(`Closed data channel with label: ${receiveChannel.label}`);
    receiveChannel = null;
  }
  localConnection.close();
  remoteConnection.close();
  localConnection = null;
  remoteConnection = null;
  console.log('Closed peer connections');

  // re-enable the file select
  fileInput.disabled = false;
  abortButton.disabled = true;
  sendFileButton.disabled = false;
}

async function gotLocalDescription(desc) {
  // local connection 的 desription 設定給 local description 以及 remote connection 的 remote description
  await localConnection.setLocalDescription(desc);
  console.log(`Offer from localConnection\n ${desc.sdp}`);
  await remoteConnection.setRemoteDescription(desc);
  try {

    // 產生 answer
    const answer = await remoteConnection.createAnswer();
    await gotRemoteDescription(answer);
  } catch (e) {
    console.log('Failed to create session description: ', e);
  }
}

async function gotRemoteDescription(desc) {
  // remote connection 的 description 設定為 local description,以及 local connection 的 remote description
  await remoteConnection.setLocalDescription(desc);
  console.log(`Answer from remoteConnection\n ${desc.sdp}`);
  await localConnection.setRemoteDescription(desc);
}

function receiveChannelCallback(event) {
  console.log('Receive Channel Callback');
  receiveChannel = event.channel;
  receiveChannel.binaryType = 'arraybuffer';
  receiveChannel.onmessage = onReceiveMessageCallback;
  receiveChannel.onopen = onReceiveChannelStateChange;
  receiveChannel.onclose = onReceiveChannelStateChange;

  receivedSize = 0;
  bitrateMax = 0;
  downloadAnchor.textContent = '';
  downloadAnchor.removeAttribute('download');
  if (downloadAnchor.href) {
    URL.revokeObjectURL(downloadAnchor.href);
    downloadAnchor.removeAttribute('href');
  }
}

function onReceiveMessageCallback(event) {
  // receive data channel 收到 16kB 資料
  console.log(`Received Message ${event.data.byteLength}`);

  // 把資料 push 到 receiveBuffer
  receiveBuffer.push(event.data);
  receivedSize += event.data.byteLength;
  receiveProgress.value = receivedSize;

  // we are assuming that our signaling protocol told
  // about the expected file size (and name, hash, etc).
  // 假設是透過 signaling protocol 取得 filename, size
  const file = fileInput.files[0];

  // 當接收到的 size 等於 file size
  if (receivedSize === file.size) {
    // 由 receiveBuffer 產生 Blob
    const received = new Blob(receiveBuffer);
    receiveBuffer = [];

    downloadAnchor.href = URL.createObjectURL(received);
    downloadAnchor.download = file.name;
    downloadAnchor.textContent =
      `Click to download '${file.name}' (${file.size} bytes)`;
    downloadAnchor.style.display = 'block';

    const bitrate = Math.round(receivedSize * 8 /
      ((new Date()).getTime() - timestampStart));
    bitrateDiv.innerHTML =
      `<strong>Average Bitrate:</strong> ${bitrate} kbits/sec (max: ${bitrateMax} kbits/sec)`;

    if (statsInterval) {
      // 清除 statsInterval
      clearInterval(statsInterval);
      statsInterval = null;
    }

    // 關閉 data channels
    closeDataChannels();
  }
}

function onSendChannelStateChange() {
  if (sendChannel) {
    const {readyState} = sendChannel;
    console.log(`Send channel state is: ${readyState}`);
    if (readyState === 'open') {

      // 當 sendChannel 的狀態改變為 open 時,就開始 sendData
      sendData();
    }
  }
}

function onError(error) {
  if (sendChannel) {
    console.error('Error in sendChannel:', error);
    return;
  }
  console.log('Error in sendChannel which is already closed:', error);
}

async function onReceiveChannelStateChange() {
  if (receiveChannel) {
    const readyState = receiveChannel.readyState;
    console.log(`Receive channel state is: ${readyState}`);
    if (readyState === 'open') {
      timestampStart = (new Date()).getTime();
      timestampPrev = timestampStart;
      statsInterval = setInterval(displayStats, 500);

      // 當 receive data channel 為 open 時,每 500ms 呼叫一次 displayStats
      await displayStats();
    }
  }
}

// display bitrate statistics.
async function displayStats() {
  if (remoteConnection && remoteConnection.iceConnectionState === 'connected') {
    // 如果 remoteConnection 還在連線中,取得 getStats
    const stats = await remoteConnection.getStats();
    let activeCandidatePair;
    stats.forEach(report => {
      if (report.type === 'transport') {
        activeCandidatePair = stats.get(report.selectedCandidatePairId);
      }
    });
    if (activeCandidatePair) {
      if (timestampPrev === activeCandidatePair.timestamp) {
        return;
      }
      // calculate current bitrate
      const bytesNow = activeCandidatePair.bytesReceived;
      const bitrate = Math.round((bytesNow - bytesPrev) * 8 /
        (activeCandidatePair.timestamp - timestampPrev));
      bitrateDiv.innerHTML = `<strong>Current Bitrate:</strong> ${bitrate} kbits/sec`;
      timestampPrev = activeCandidatePair.timestamp;
      bytesPrev = bytesNow;
      if (bitrate > bitrateMax) {
        bitrateMax = bitrate;
      }
    }
  }
}

Transfer data


    <section>
        <div id="button">
            <button id="sendTheData" type="button">Generate and send data</button>
        </div>
        <div class="input">
            <input type="number" id="megsToSend" min="1" name="megs" value="16"/>
            <label for="megsToSend">MB <b>(warning: very large values will potentially cause memory problems)</b></label>
            <div id="errorMsg"></div>
        </div>
        <div class="input">
            <input type="checkbox" id="ordered" checked>
            <label for="ordered">Ordered mode</label>
        </div>
        <div class="progress">
            <div class="label">Send progress:</div>
            <progress id="sendProgress" max="0" value="0"></progress>
        </div>

        <div class="progress">
            <div class="label">Receive progress:</div>
            <progress id="receiveProgress" max="0" value="0"></progress>
        </div>

        <div>
            <span id="transferStatus"></span>
        </div>
    </section>
/*
 *  Copyright (c) 2015 The WebRTC project authors. All Rights Reserved.
 *
 *  Use of this source code is governed by a BSD-style license
 *  that can be found in the LICENSE file in the root of the source
 *  tree.
 */

'use strict';
// 256k = 256 * 1024
const MAX_CHUNK_SIZE = 262144;

let localConnection;
let remoteConnection;
let sendChannel;
let receiveChannel;
// chunkSize = Math.min(localConnection.sctp.maxMessageSize, MAX_CHUNK_SIZE);
let chunkSize;
// lowWaterMark = chunkSize; // A single chunk
let lowWaterMark;
let highWaterMark;
let dataString;
let timeoutHandle = null;
const megsToSend = document.querySelector('input#megsToSend');
const sendButton = document.querySelector('button#sendTheData');
const orderedCheckbox = document.querySelector('input#ordered');
const sendProgress = document.querySelector('progress#sendProgress');
const receiveProgress = document.querySelector('progress#receiveProgress');
const errorMessage = document.querySelector('div#errorMsg');
const transferStatus = document.querySelector('span#transferStatus');

let bytesToSend = 0;
let totalTimeUsedInSend = 0;
let numberOfSendCalls = 0;
let maxTimeUsedInSend = 0;
let sendStartTime = 0;
let currentThroughput = 0;

sendButton.addEventListener('click', createConnection);

// Prevent data sent to be set to 0.
megsToSend.addEventListener('change', function() {
  const number = this.value;
  if (Number.isNaN(number)) {
    errorMessage.innerHTML = `Invalid value for MB to send: ${number}`;
  } else if (number <= 0) {
    sendButton.disabled = true;
    errorMessage.innerHTML = '<p>Please enter a number greater than zero.</p>';
  } else if (number > 64) {
    // 限制小於 64 MB
    sendButton.disabled = true;
    errorMessage.innerHTML = '<p>Please enter a number lower or equal than 64.</p>';
  } else {
    errorMessage.innerHTML = '';
    sendButton.disabled = false;
  }
});

async function createConnection() {
  sendButton.disabled = true;
  megsToSend.disabled = true;

  const servers = null;

  // 計算要發送的資料量
  const number = Number.parseInt(megsToSend.value);
  bytesToSend = number * 1024 * 1024;

  // 產生 local peer connection
  localConnection = new RTCPeerConnection(servers);

  // 產生 local send data channel, 且設定 ordered 參數
  // Let's make a data channel!
  const dataChannelParams = {ordered: false};
  if (orderedCheckbox.checked) {
    dataChannelParams.ordered = true;
  }
  sendChannel = localConnection.createDataChannel('sendDataChannel', dataChannelParams);
  sendChannel.addEventListener('open', onSendChannelOpen);
  sendChannel.addEventListener('close', onSendChannelClosed);
  console.log('Created send data channel: ', sendChannel);

  console.log('Created local peer connection object localConnection: ', localConnection);

  // icecandidate callback
  localConnection.addEventListener('icecandidate', e => onIceCandidate(localConnection, e));

  /////////
  // remote peer connection
  remoteConnection = new RTCPeerConnection(servers);
  remoteConnection.addEventListener('icecandidate', e => onIceCandidate(remoteConnection, e));
  remoteConnection.addEventListener('datachannel', receiveChannelCallback);

  /////////
  try {
    // local peer connection 產生 offer
    const localOffer = await localConnection.createOffer();
    await handleLocalDescription(localOffer);
  } catch (e) {
    console.error('Failed to create session description: ', e);
  }

  transferStatus.innerHTML = 'Peer connection setup complete.';
}

function sendData() {
  // Stop scheduled timer if any (part of the workaround introduced below)
  if (timeoutHandle !== null) {
    clearTimeout(timeoutHandle);
    timeoutHandle = null;
  }

  let bufferedAmount = sendChannel.bufferedAmount;
  while (sendProgress.value < sendProgress.max) {
    transferStatus.innerText = 'Sending data...';
    const timeBefore = performance.now();

    sendChannel.send(dataString);

    const timeUsed = performance.now() - timeBefore;
    if (timeUsed > maxTimeUsedInSend) {
      maxTimeUsedInSend = timeUsed;
      totalTimeUsedInSend += timeUsed;
    }
    numberOfSendCalls += 1;
    bufferedAmount += chunkSize;
    sendProgress.value += chunkSize;

    // Pause sending if we reach the high water mark
    if (bufferedAmount >= highWaterMark) {
      // This is a workaround due to the bug that all browsers are incorrectly calculating the
      // amount of buffered data. Therefore, the 'bufferedamountlow' event would not fire.
      if (sendChannel.bufferedAmount < lowWaterMark) {
        timeoutHandle = setTimeout(() => sendData(), 0);
      }
      console.log(`Paused sending, buffered amount: ${bufferedAmount} (announced: ${sendChannel.bufferedAmount})`);
      break;
    }
  }

  if (sendProgress.value === sendProgress.max) {
    transferStatus.innerHTML = 'Data transfer completed successfully!';
  }
}

function startSendingData() {
  // 開始發送 data
  transferStatus.innerHTML = 'Start sending data.';
  sendProgress.max = bytesToSend;
  receiveProgress.max = sendProgress.max;
  sendProgress.value = 0;
  receiveProgress.value = 0;
  sendStartTime = performance.now();
  maxTimeUsedInSend = 0;
  totalTimeUsedInSend = 0;
  numberOfSendCalls = 0;
  sendData();
}

function maybeReset() {
  if (localConnection === null && remoteConnection === null) {
    sendButton.disabled = false;
    megsToSend.disabled = false;
  }
}

async function handleLocalDescription(desc) {
  // 處理 local peer connection 的 sdp description
  localConnection.setLocalDescription(desc);
  console.log('Offer from localConnection:\n', desc.sdp);
  remoteConnection.setRemoteDescription(desc);
  try {
    // 產生 answer
    const remoteAnswer = await remoteConnection.createAnswer();
    handleRemoteAnswer(remoteAnswer);
  } catch (e) {
    console.error('Error when creating remote answer: ', e);
  }
}

function handleRemoteAnswer(desc) {
  remoteConnection.setLocalDescription(desc);
  console.log('Answer from remoteConnection:\n', desc.sdp);
  localConnection.setRemoteDescription(desc);
}

function getOtherPc(pc) {
  return (pc === localConnection) ? remoteConnection : localConnection;
}

async function onIceCandidate(pc, event) {
  const candidate = event.candidate;
  if (candidate === null) {
    return;
  } // Ignore null candidates
  try {
    await getOtherPc(pc).addIceCandidate(candidate);
    console.log('AddIceCandidate successful: ', candidate);
  } catch (e) {
    console.error('Failed to add Ice Candidate: ', e);
  }
}

function receiveChannelCallback(event) {
  console.log('Receive Channel Callback');
  receiveChannel = event.channel;
  receiveChannel.binaryType = 'arraybuffer';
  receiveChannel.addEventListener('close', onReceiveChannelClosed);
  receiveChannel.addEventListener('message', onReceiveMessageCallback);
}

function onReceiveMessageCallback(event) {
  // 在 receiveChannel 收到 message
  receiveProgress.value += event.data.length;
  currentThroughput = receiveProgress.value / (performance.now() - sendStartTime);
  console.log('Current Throughput is:', currentThroughput, 'bytes/sec');

  // Workaround for a bug in Chrome which prevents the closing event from being raised by the
  // remote side. Also a workaround for Firefox which does not send all pending data when closing
  // the channel.
  if (receiveProgress.value === receiveProgress.max) {
    sendChannel.close();
    receiveChannel.close();
  }
}

function onSendChannelOpen() {
  // sendChannel 成功 open
  console.log('Send channel is open');

  chunkSize = Math.min(localConnection.sctp.maxMessageSize, MAX_CHUNK_SIZE);
  console.log('Determined chunk size: ', chunkSize);
  dataString = new Array(chunkSize).fill('X').join('');
  lowWaterMark = chunkSize; // A single chunk
  highWaterMark = Math.max(chunkSize * 8, 1048576); // 8 chunks or at least 1 MiB
  console.log('Send buffer low water threshold: ', lowWaterMark);
  console.log('Send buffer high water threshold: ', highWaterMark);
  sendChannel.bufferedAmountLowThreshold = lowWaterMark;

  sendChannel.addEventListener('bufferedamountlow', (e) => {
    console.log('BufferedAmountLow event:', e);
    sendData();
  });

  startSendingData();
}

function onSendChannelClosed() {
  console.log('Send channel is closed');
  localConnection.close();
  localConnection = null;
  console.log('Closed local peer connection');
  maybeReset();
  console.log('Average time spent in send() (ms): ' +
              totalTimeUsedInSend / numberOfSendCalls);
  console.log('Max time spent in send() (ms): ' + maxTimeUsedInSend);
  const spentTime = performance.now() - sendStartTime;
  console.log('Total time spent: ' + spentTime);
  console.log('MBytes/Sec: ' + (bytesToSend / 1000) / spentTime);
}

function onReceiveChannelClosed() {
  console.log('Receive channel is closed');
  remoteConnection.close();
  remoteConnection = null;
  console.log('Closed remote peer connection');
  maybeReset();
}

Trickle ICE

Trickle ICE

如果是 stun server,candidate 必須要取得 srflx

如果是 turn server,candidate 必須要取得 relay

Time Component Type Foundation Protocol Address Port Priority
0.004 rtp host 3868393361 udp 192.168.1.157 54299 126 | 32542 | 255
0.008 rtp srflx 1742403877 udp 220.132.127.162 64347 100 | 32542 | 255
0.064 rtp relay 2774939105 udp 211.72.214.206 41556 2 | 32542 | 255
0.107 rtp host 2819687265 tcp 192.168.1.157 9 90 | 32542 | 255
0.107 Done
0.109

References

WebRTC samples

2021/09/27

WebRTC API

Web Real-Time Communication API 是提供網頁進行視訊或語音串流通訊的技術,可以直接使用瀏覽器進行資料交換。WebRTC 可提供瀏覽器在不需要安裝外掛程式或第三方軟體下,分享應用程式的資料和進行電話會議。

因為 WebRTC 還在發展當中,各 browser 有不同程度的 codec 支援以及 WebRTC 功能。故建議使用 adapter ,這是 Google 提供,減少跨瀏覽器差異的 library。

browser compatability matrix 是不同瀏覽器的相容表。

WebRTC 有多種功能,包含 audio, video conferencing,檔案交換,screen sharing,identity management,支援傳統交換機的 DTMF signals,兩端點之間的連線不需要特殊的 drivers/plugins,也可不需要中間 server。

兩個 peers 之間是以 RTCPeerConnection 產生連線,當連線建立後,就能夠使用 MediaStream 或 RTCDataChannel。

MediaStream 包含了數個 media information tracks,是由 MediaStreamTrack 介面表示的物件,可包含多種 media data: audio, video, text,大部分的 streams 包含至少一條 audio track,一條 video track,可用來傳送 live media,或將 media 存檔。

也可以用 RTCDataChannel 介面在兩端之間,傳遞任意的 binray data,這邊可用來傳送 channel-information, 交換 metadata,game status, file transfer。

WebRTC References

Connection setup and management

Interfaces

Interfaces Description
RTCPeerConnection local 與 remote peer 之間的 WebRTC connection
RTCDataChannel connection 的兩個 peers 之間的 bi-directional data channel
RTCDataChannelEvent 將 RTCDataChannel 接到 RTCPerrConnection 的 event
RTCSessionDescription session 的參數,裡面包含了 description type,代表它所描述的部分 offer/answer negotiation process 以及該 seesion的 SDP descriptor
RTCStatsReport connection 或是 tack on connection 的統計資訊。以 RTCPeerConnection.getStats() 取得。使用 WebRTC 的統計資料有另外的 WebRTC Statistics API
RTCIceCandidate 代表一個可用來建立 RTCPeerConnection 的 candidate ICE server
RTCIceTransport ICE transport 的資訊
RTCPeerConnectionIceEvent 在 RTCPeerConnection 中,有關 ICE candidates 的相關 events。event 是使用 type: icecandidate
RTCRtpSender 在 RTCPeerConnection 裡面的 MediaStreamTrack,用來encoding 傳輸資料
RTCRtpReceiver 在 RTCPeerConnection 裡面的 MediaStreamTrack,用來decoding 傳輸資料
RTCTrackEvent track event 介面。通常是 RTCRtpReceiver 接到 RTCPeerConnection 時發生,代表 RTCPeerConenction 產生了新的 MediaStreamTrack
RTCSctpTransport SCTP transport 的資訊,可在 RTCPeerConnection 的 data channel 收送資料時,透過 SCTP packets,存取Datagram Transport Layer Security(DTLS)

Dictionaries

Dictionaries Description
RTCConfiguration RTCPeerConnection 的 conifguration options
RTCIceServer 定義如何連接 ICE Sever,例如 Stun or Turn server
RTCRtpContributingSource 有關 contributing source (SCRC) 的資訊,包含該 source 被播放了一個最新的 packet 的 most recent time

ex:

var configuration = { iceServers: [{
                          urls: "stun:stun.services.mozilla.com",
                          username: "louis@mozilla.com",
                          credential: "webrtcdemo"
                      }, {
                          urls: ["stun:stun.example.com", "stun:stun-1.example.com"]
                      }]
};

var pc = new RTCPeerConnection(configuration);

Events

Events Description
bufferedamountlow data channel 目前 buffered 的資料量,也就是 bufferedAmount property 已經低於 bufferedAmountLowThreshold 這個 minimum buffered data size
close data channel 已結束 process,進入 closed state,data transport 已經完全 closed,在沒有完全 closed 的時候,會發送 closing event
closing RTCDataChannel 已轉換到 closing state,如果完全停止則是 close event
connectionstatechange 連線狀態(connectionState) 改變
datachannel 有新的 RTCDataChannel 可以使用,事件類別為 RTCDataChannelEvent
error data channel 發生錯誤 RTCErrorEvent
error RTCDtlsTransport 發生錯誤 RTCErrorEvent,可能是 dtls-failure 或是 fingerprint-failure
gatheringstatechange RTCIceTransport 的 gathering state 已改變
icecandidate RTCPeerConnectonIceEvent
當 local device 識別到一個新的 ICE candidate,並需要 local peer 去呼叫 setLocalDescription() 時,會產生此 event
icecandidateerror RTCPeerConnectionIceErrorEvent
在 gathering ICE candidates 發生錯誤時
iceconnectionstatechange 當 ICE gathering state (屬性 icegatheringstate) 改變時,會發送給 RTCPeerConnection
message data channel 收到一個訊息,event type 為 MessageEvent
negotiationneeded 通知 RTCPeerconnection 需要處理 session negotiation,可在 setLocalDescription() 後,呼叫 createOffer()
open RTCDataChannel 的 data transport 已經 opened 或 re-opened
selectedcandidatepairchane RTCIceTransport 目前選用的 ICE candidates 改變
track 當新的 track 在成功協調 media streaming 後,會產生 RTCTrackevent 發送給 RTCPeerConnection
signalingstatechange 當 signalingstate 改變時,送給 peer connection。這會發生在呼叫 setLocalDescription() 或 setRemoteDescription() 之後
statechange RTCDtlsTransport 的狀態改變
statechange RTCIceTransport 的狀態改變
statechange RTCSctpTransport 的狀態改變

Types

Types Description
RTCSctpTransport.state 代表 RTCSctpTrnasport instance 的 state
RTCSessionDescriptionCallback 當要求 create offers 或 answers 時,RTCSessionDescriptionCallback 會傳給 RTCPeerConnection

Identity and security

管理 identity and security,用在連線的認證

identity Description
RTCIdentityProvider user agent,可要求處理 identity assertion
RTCIdentityAssertion 代表 current connection 的 identity of the remote peer。如果目前沒有 peer,就會回傳 null,一但被設定,就不能被修改
RTCIdentityProviderRegistrar Registers and identity provider (idP)
RTCIdentityEvent 代表 identity provider (idP) 產生了 identity assertion。event type 為 identityresult
RTCIdentityErrorEvent 代表 identity provider (idP) 產生了 error。event type 為 idpassertionerror 或 idpvalidationerror
RTCertificate RTCPeerConnection 用來認證的 certificate

Telephony

PSTN 使用的 dialing tone

Interfaces

Interfaces Description
RTCDTMFSender 發送 DTMF signaling
RTCDTMFToneChangeEvent tonechange event 用來表示 DTMF 開始或結束

Events

Events Description
tonechange 當connection 開始播放 DTMF tone,或是 RTCDTMFSender 的 toneBuffer 已經送完時,會產生 event type: RTCDTMFToneChangeEvent

WebRTC Connectivity

Signaling

WebRTC 必須搭配 signaling service 才能使用。用來交換 SDP 資訊,也就是 offer/answer 資料

  • Peer A init connection 產生 Offer
  • 透過 signal channel 發送 Offer 給 Peer B
  • Peer B 接收 Offer,產生 Answer
  • 透過 signal channel 回送 Answer

Session Descriptions

configuration of an endpoint on a WebRTC connection 就稱為 session description

以 SDP 描述,裡面包含 media type, format, transfer protocol, endpoint's IP/Port, 其他資訊

media data exchange 是以 Interactive Connectivity Establishment (ICE) 處理,該協定可讓兩個在 NAT 下的 devices 使用 intermediary 交換 offers, answers

每個 peer 都留存兩個 description:localdescription 及 remote description

當建立 call 或是 某一端要修改 configuration 時,就會透過 offer/answer process 處理,以下是基本的 steps

  1. caller 透過 MediaDevices.getUserMedia 取得 local Media
  2. caller 產生 RTCPeerConnection,呼叫 RTCPeerConnection.addTrack()
  3. caller 呼叫 RTCPeerConnection.createOffer() 產生 offer
  4. caller 呼叫 RTCPeerConnection.setLocalDescription() 發送 offer 到 local description
  5. caller 詢問 STUN servers 用以產生 ice candidates
  6. caller 使用 signaling server 傳送 offer 給 receiver
  7. receiver 接收 offer,呼叫 RTCPeerConnection.setRemoteDescription() 作為 remote description
  8. receiver 設定 call
    • capture local media
    • 呼叫 RTCPeerConnection.addTrack()
  9. receiver 透過 RTCPeerConnection.createAnswer() 產生 answer
  10. receiver 呼叫 RTCPeerConnection.setLocalDescription(),傳入 answer,也就是將 answer 設定為 local description
  11. receiver 使用 signaling server 發送 answer 給 caller
  12. caller 收到 answer
  13. caller 呼叫 RTCPeerConnection.setRemoteDescription() 將 answer 設定為 remote description
  14. 然後就能開始互傳 media

Pending and Current Descriptions

在處理 localDescription 與 remoteDescription 時,可能會因為格式不相容而被 reject,因此需要一個機制能夠 propose a new format,直到另一端接受

current description:由 RTCPeerConnection.currentLocalDescription 及 RTCPeerConenction.currentRemoteDescription 回傳,這也是目前雙方同意使用的 description。

pending description:由 RTCPeerConnection.pendingLocalDescription 及 RTCPeerConnection.pendingRemoteDescription 回傳,代表 setLocalDesciption() 或 setRemoteDescription() 目前考慮中的 description

RTCPeerConnection.localDescription 與 RTCPeerConnection.remoteDescription 回傳的 description,如果在 pending description 狀態,就是回傳 pending description,否則就回傳 current description

呼叫 setLocalDescription() 或 setRemoteDescription() 時,description 會先設定為 pending description,然後 WebRTC layer 開始評估要不要接受,一但 proposed description 被接受,current description 就會改成 pending description 的值,pending description 被改為 null

pendingLocalDescription 包含 offer or answer,還有 local ICE candidates

pendingRemoteDescription 包含 remote ICE candidates,可呼叫 RTCPeerconnection.addIceCandidate() 提供資訊

ICE Candidates

除了交換 media information,還需要交換 network connection 的 informations,也就是 ICE candidate 以及 peer 能夠通訊的方法 (directly or through a TURN server)

通常每個 peer 會先 propose 最佳的 candidates,理想狀態要使用 UDP (速度快,且容易 recover 中斷),但 ICE 標準也允許使用 TCP (當無法使用 UDP 時),另外不是所有瀏覽器都支援 ICE over TCP

UDP candidate types

  • host:實體 IP,可直接連線

  • srflx:Server Reflexive and Relayed Candidates

    host 因為經過 NAT,透過 Stun/Turn server 產生的 candidate。connection 最初要求來自 STUN server 的 candidate,並將 request 傳給 remote peer

  • prflx: Peer Reflexive Candidates

    來自 symmetric NAT,通常是 trickle ICE (也就是 additional candidate 交換,發生在 primary signaling 之後,在 verification 結束以前) 的 additional candidate

    是直接發送給 peer host 產生的 candidate

  • relay

    類似 srflx,但是由 Turn Server 產生的中繼 candidate IP:Port

tcp 有三種:active, passive, so

  • active

    會打開 outbound connection,且不接受 incoming connection request。這是最常見的類型

  • passive

    只會接受 incoming connection

  • so

    會嘗試同時在兩個終端直接打開連線

Choosing a candidate pair

ICE layer 會選擇兩個 peers 中的某一個當作 controlling agent。ICE Agent 會決定 connection 最後使用的 candidate pair,另一個 peer 就稱為 controlled agent。可透過 RTCIceCandidate.transport.role 查詢

controlling agent 除了決定 candidate pair 以外,還會透過 STUN,signaling 選用的 controlled agent,更新 offer。controlled agent 只會等待通知使用哪一個 candidate pair

ICE session 可能會讓 controlling agent 選擇多個 candidate pair,每次分享資訊給 controlled agent 時,兩端會重新設定 connection 使用的 candidate pair

一但 ICE session 完成,configuration 就固定了,除非發生 ICE reset

在每個 candidate generation 結束時,會以 RTCIceCandidate 格式發送 end-of-candidates notification,也就是 candidate 屬性為空字串。這個 candidate 還是要呼叫 addIceCandidate() 加入 connection,以便通知 remote peer

如果在 negotiation 時,沒有其他的 candidates,就會發送 end-of-candidates notification,發送 candidate 屬性為 null 的 RTCIceCandidate。這個訊息不需要傳給 remote peer。這是 legacy notification of a state,可用 icegatheringstatechange event 偵測得知

ICE rollbacks

如果不像掛斷既有的 call,可作 renegotiation

ICE rollback 會 restore 上次連線中 signalingState 為 stable 的 SDP offer

發送 description 其 type 為 rollback,就可以 init rollback,該 description 的其他 properties 都會被忽略

當 peer 已經有先前產生的 offer 時,ICE agent 會自動產生 rollback。如果 local peer 的狀態為 have-local-offer,也就是 local pper 有發送過 offer

呼叫 setRemoteDescription() 可收到 received offer triggers rollback

完整的 exchange


Signaling and video calling

Signaling and video calling

discovery 與 negotiation process 用以產生 conenction,稱為 signaling

WebRTC 需要兩個不同網路的端點,能夠互相找到對方,並協調 media format,需要透過第三方 server 處理 signaling

signaling server

兩個 devices 之間的 WebRTC 連線需要先透過 signaling server 協調兩端

WebRTC 不指定 transport mechanism,可使用 WebSocket 或是 XMLHttpRequest

server 不需要了解或是處理 signaling data content (SDP),只需要傳遞。ICE subsystem 要如何傳送 singaling data 給另一個 peer 是比較重要的問題。

準備給 signaling 使用的 chat server

chat server 透過 WebSocket 傳送 JSON string

可用類似方法,做 signaling 與 ICE negotiation

# 將訊息傳送給特定 target 的 function
function sendToOneUser(target, msgString) {
  var isUnique = true;
  var i;

  for (i=0; i < connectionArray.length; i++) {
    if (connectionArray[i].username === target) {
      connectionArray[i].send(msgString);
      break;
    }
  }
}

Designing the signaling protocol

需要一個 protocol 交換 message,以下利用 JSON object 作為範例

Exchange session descriptions

在開始 singaling process 時,user init call 並產生 offer,裡面包含 session description (SDP 格式),該資料要傳給 callee,callee 會以 answer 回應

設計用 video-offer video-answer 分別代表 offer, answer

JSON 欄位 說明
type message type, video-offer or video-answer
name sender's name
target callee's name
sdp SDP string

兩端可知道 codecs, codec parameters,但還不知道如何傳遞 media data。透過 ICE candidates 處理

Exchange ICE candidates

兩端需要交換 ICE candidates 協調真正的 connection,每個 ICE candidate 都會描述一個可使用的溝通管道。peer 會持續發送找到的 candidates,不管 streaming 是否已經開始了。

icecandidate event 會送給 RTCPeerConenction,並以 pc.setLocalDescription(offer) 完成 process

如果有更好的 candidate,stream 可修改 formats

欄位 說明
type new-ice-candidate
target callee's name
candidate SDP candidate string
不需要處理訊息內容,只需要傳給 remote peer

每個 ICE message 會提供 TCP/UDP, IP address, port, connection type (說明是 peer IP 或是 relay server),包含 NAT 資訊

note: 當收到 onicecandidate 時,要將 candidate 透過 signaling connection 傳給另一個 peer。收到 ICE candidate message 時,就呼叫 RTCPeerConnection.addIceCandidate()。建議不要嘗試修改 SDP

note: onicecandidate event 與 createAnswer() 都是 async call,注意 signaling 不要修改處理順序。setRemoteDescription() 後,才能呼叫 addIceCandidate()

Signaling transaction flow

  • Each user's client running within a web browser
  • Each user's web browser
  • The signaling server
  • The web server hosting the chat service

ICE candidate exchange process

在 local ICE layer 收到的 candidate,各自發送給另一端,當兩端同意使用某個 candidate,就會開始傳遞 media

當網路狀況有異動,peer 可能會建議切換到不同 media resolution 或 不同 codec,就會啟動新的 exchange of candidates

client application

兩個 video elements,一個 hangup 按鈕

<div class="flexChild" id="camera-container">
  <div class="camera-box">
    <video id="received_video" autoplay></video>
    <video id="local_video" autoplay muted></video>
    <button id="hangup-button" onclick="hangUpCall();" disabled>
      Hang Up
    </button>
  </div>
</div>
// 透過 WebSocket 發送訊息給 signaling server
function sendToServer(msg) {
  var msgJSON = JSON.stringify(msg);

  connection.send(msgJSON);
}
// users: 每個連線的 user 的 usernames
function handleUserlistMsg(msg) {
  var i;
  var listElem = document.querySelector(".userlistbox");

  while (listElem.firstChild) {
    listElem.removeChild(listElem.firstChild);
  }

  msg.users.forEach(function(username) {
    var item = document.createElement("li");
    item.appendChild(document.createTextNode(username));
    item.addEventListener("click", invite, false);

    listElem.appendChild(item);
  });
}
// 當 user click 某個想要呼叫的 username,就呼叫 inivte
var mediaConstraints = {
  audio: true, // We want an audio track
  video: true // ...and we want a video track
};

function invite(evt) {
  if (myPeerConnection) {
    alert("You can't start a call because you already have one open!");
  } else {
    var clickedUsername = evt.target.textContent;

    if (clickedUsername === myUsername) {
      alert("I'm afraid I can't let you talk to yourself. That would be weird.");
      return;
    }

    targetUsername = clickedUsername;
    createPeerConnection();

    navigator.mediaDevices.getUserMedia(mediaConstraints)
    .then(function(localStream) {
      document.getElementById("local_video").srcObject = localStream;
      localStream.getTracks().forEach(track => myPeerConnection.addTrack(track, localStream));
    })
    .catch(handleGetUserMediaError);
  }
}
// 如果 getUserMedia 發生 error 的錯誤處理
function handleGetUserMediaError(e) {
  switch(e.name) {
    case "NotFoundError":
      alert("Unable to open your call because no camera and/or microphone" +
            "were found.");
      break;
    case "SecurityError":
    case "PermissionDeniedError":
      // Do nothing; this is the same as the user canceling the call.
      break;
    default:
      alert("Error opening your camera and/or microphone: " + e.message);
      break;
  }

  closeVideoCall();
}
// 建立 PeerConnection
function createPeerConnection() {
  myPeerConnection = new RTCPeerConnection({
      iceServers: [     // Information about ICE servers - Use your own!
        {
          urls: "stun:stun.stunprotocol.org"
        }
      ]
  });

  myPeerConnection.onicecandidate = handleICECandidateEvent;
  myPeerConnection.ontrack = handleTrackEvent;
  myPeerConnection.onnegotiationneeded = handleNegotiationNeededEvent;
  myPeerConnection.onremovetrack = handleRemoveTrackEvent;
  myPeerConnection.oniceconnectionstatechange = handleICEConnectionStateChangeEvent;
  myPeerConnection.onicegatheringstatechange = handleICEGatheringStateChangeEvent;
  myPeerConnection.onsignalingstatechange = handleSignalingStateChangeEvent;
}
  • Starting negotiation
// 處理 negotiationneeded event
function handleNegotiationNeededEvent() {
  myPeerConnection.createOffer().then(function(offer) {
    return myPeerConnection.setLocalDescription(offer);
  })
  .then(function() {
    sendToServer({
      name: myUsername,
      target: targetUsername,
      type: "video-offer",
      sdp: myPeerConnection.localDescription
    });
  })
  .catch(reportError);
}
  • session negotiation
    • Handling the invitation
function handleVideoOfferMsg(msg) {
  var localStream = null;

  targetUsername = msg.name;
  createPeerConnection();

  var desc = new RTCSessionDescription(msg.sdp);

  myPeerConnection.setRemoteDescription(desc).then(function () {
    return navigator.mediaDevices.getUserMedia(mediaConstraints);
  })
  .then(function(stream) {
    localStream = stream;
    document.getElementById("local_video").srcObject = localStream;

    localStream.getTracks().forEach(track => myPeerConnection.addTrack(track, localStream));
  })
  .then(function() {
    return myPeerConnection.createAnswer();
  })
  .then(function(answer) {
    return myPeerConnection.setLocalDescription(answer);
  })
  .then(function() {
    var msg = {
      name: myUsername,
      target: targetUsername,
      type: "video-answer",
      sdp: myPeerConnection.localDescription
    };

    sendToServer(msg);
  })
  .catch(handleGetUserMediaError);
}
  • sending ICE candidates
function handleICECandidateEvent(event) {
  if (event.candidate) {
    sendToServer({
      type: "new-ice-candidate",
      target: targetUsername,
      candidate: event.candidate
    });
  }
}
  • receiving ICE candidates
function handleNewICECandidateMsg(msg) {
  var candidate = new RTCIceCandidate(msg.candidate);

  myPeerConnection.addIceCandidate(candidate)
    .catch(reportError);
}
  • receiving new streams
function handleTrackEvent(event) {
  document.getElementById("received_video").srcObject = event.streams[0];
  document.getElementById("hangup-button").disabled = false;
}
  • handling the removal of tracks
function handleRemoveTrackEvent(event) {
  var stream = document.getElementById("received_video").srcObject;
  var trackList = stream.getTracks();

  if (trackList.length == 0) {
    closeVideoCall();
  }
}
  • hangup
function hangUpCall() {
  closeVideoCall();
  sendToServer({
    name: myUsername,
    target: targetUsername,
    type: "hang-up"
  });
}
  • end call
function closeVideoCall() {
  var remoteVideo = document.getElementById("received_video");
  var localVideo = document.getElementById("local_video");

  if (myPeerConnection) {
    myPeerConnection.ontrack = null;
    myPeerConnection.onremovetrack = null;
    myPeerConnection.onremovestream = null;
    myPeerConnection.onicecandidate = null;
    myPeerConnection.oniceconnectionstatechange = null;
    myPeerConnection.onsignalingstatechange = null;
    myPeerConnection.onicegatheringstatechange = null;
    myPeerConnection.onnegotiationneeded = null;

    if (remoteVideo.srcObject) {
      remoteVideo.srcObject.getTracks().forEach(track => track.stop());
    }

    if (localVideo.srcObject) {
      localVideo.srcObject.getTracks().forEach(track => track.stop());
    }

    myPeerConnection.close();
    myPeerConnection = null;
  }

  remoteVideo.removeAttribute("src");
  remoteVideo.removeAttribute("srcObject");
  localVideo.removeAttribute("src");
  remoteVideo.removeAttribute("srcObject");

  document.getElementById("hangup-button").disabled = true;
  targetUsername = null;
}

ICE connection state

function handleICEConnectionStateChangeEvent(event) {
  switch(myPeerConnection.iceConnectionState) {
    case "closed":
    case "failed":
      closeVideoCall();
      break;
  }
}

ICE signaling state

function handleSignalingStateChangeEvent(event) {
  switch(myPeerConnection.signalingState) {
    case "closed":
      closeVideoCall();
      break;
  }
};

ICE gathering state

function handleICEGatheringStateChangeEvent(event) {
  // Our sample just logs information to console here,
  // but you can do whatever you need.
}

WebRTC Data Channel

Using WebRTC Data Cahnnel

建立 RTCPeerConnection 後,就能收送 media data,也可以使用 data channel,可安全地交換任何資料

因所有 WebRTC components 都要求加密,RTCDataChannel 會自動使用 Datagram Transport Layer Security (DTLS)

產生 data channel

RTCDataChannel 使用的 data transport 可用兩種方法產生

  • 以 WebRTC 產生 transport,並宣告為 remote peer (會收到 datachannel event),這種方法比較簡單且常用,但比較沒有彈性
  • 自己寫 code 協調 data transport,自己 signal 到另一端 peer,且要連接到 new channel

Automatic negotiation

通常可讓 peer connection 協調處理 RTCDataChannel connection

呼叫 createDataChannel() 且不設定 negotiated 屬性,或是設定為 false。就會讓 RTCPeerConnection 自動處理 RTCDataChannel

可透過傳給 RTCDataChannel 的 open event 判斷是否有成功連線

let dataChannel = pc.createDataChannel("MyApp Channel");

dataChannel.addEventListener("open", (event) => {
  beginTransmission(dataChannel);
});

Manual negotiation

先用 RTCPeerConnection.createDataChannel() 產生一個新的 RTCDataChannel,設定 negotiated 為 true

使用 webserver or others,要能 signal to the remote peer,並讓他產生自己的 RTCDataChannel,且 negotiated 為 true

let dataChannel = pc.createDataChannel("MyApp Channel", {
  negotiated: true
});

dataChannel.addEventListener("open", (event) => {
  beginTransmission(dataChannel);
});

requestRemoteChannel(dataChannel.id);

上面的 requestRemoteChannel 就是用來 trigger negotiation,產生 remote channel with the same ID as the local channel

Buffering

WebRTC data channel 支援 outbound data 的 buffering,會自動處理,且無法控制 buffer size。

使用 bufferedAmount, bufferedAmountLowThreshold, onbufferedamountlow, and bufferedamountlow

了解 message size limits

網路傳輸的封包有 size 限制

Firefox, Chrome 使用 usrsctp library 實作 SCTP,但兩個使用方法不同,firefox 會用多個 SCTP message 傳送 large message (deprecated),chrome 是用 series of messages

少於 16 kB 的訊息不需要考慮此問題

Concerns

目前不適合使用 RTCDataChannel 傳送超過 64kB 的 message,因為 SCTP 最初用在 signaling protocol,這邊假設 message 都很小,要支援超過網路 MTU 的 message 是後來增加的功能。這個技術需要訊息編上有順序的號碼,要依序傳送。

因為有越多越多 application 傳送大 message,造成重要的 signaling message 阻塞的問題。

目前 browser 利用 end-of-record (EOR) flag 支援 larger message,這表示 message 是 last one in a series。在支援 EOR 的狀況下,RTCDataChannel 可傳送大 message (可超過 256 kB,firefox 實作可傳送 1GB),在 256 kB 的狀況下,會發生 noticeable delays。

要解決此問題,新的 stream scheduler (SCTP ndata specification) 可 interleave 在不同 stream 傳送的 message,proposal 目前還在 IETF draft,如果實作,理論上可傳送任意大小的 message

firefox 在 bug 1381145 支援 ndata

chrome 在 bug 5696

Security

透過 WebRTC 傳送的資料都會被加密,RTCDataChannel 是用 Datagram Transport Layer Security (DTLS) 加密,based on TLS (https)

WebRTC 在兩個 agents 之間做 peer-to-peer conenction,資料不會傳給 web/application server

SCTP NAT

解决iptables nat sctp协议无效的问题

A----->B-----C

IP如下:

A:1.1.1.1

B:1.1.1.2; 2.2.2.1

C:2.2.2.2

需求為,A 需要使用sctp連通C

在B機器上添加iptables規則為:

iptables -t nat -I PREROUTING -d 1.1.1.2 -p sctp --dport 11111 -j DNAT --to-destination  2.2.2.2:11111

就是把A發出的目的地址:Port 由1.1.1.2:11111轉變為2.2.2.2:11111

同時在C的接口上使用tcpdump抓包,發現並沒有接收到sctp message,為檢驗網絡是否正常(包括路由等配置),僅將上述規則中的sctp改為tcp進行tcp的連通測試

iptables -t nat -I PREROUTING -d 1.1.1.1 -p tcp --dport 11111 -j DNAT --to-destination  2.2.2.2:11111

發現C機器上可以抓到tcp報文,說明網絡沒有問題,iptables的規則也沒有問題。

因為tcpdump抓包解包並不需要系統支持特定的協議,懷疑可能是iptables規則因為某種原因沒有生效,借助google發現瞭解決辦法:iptables-nat-not-work-for-sctp

載入nfconntrackproto_sctp即可,該模塊用來對sctp進行連接跟蹤

modprobe nf_conntrack_proto_sctp

連接跟蹤模塊可以參見:nf_conntrack連接跟蹤模塊

Why is SCTP not much used/known

  • NAT: Doesn't cross NAT very well/at all (less than 1% internet home & enterprise routers do NAT on SCTP).

sctp over udp encapsulation does not work under NAT network


ORTC

Object Real-Time Communication 是新的規格,提供開發 WebRTC application 高階 API,ORTC 不使用 SDP,也不支援 Offer/Answer state machine。ORTC 使用 "sender", "receiver", "transport" 物件及設定參數,用以描述該物件的功能。"Tracks" 是由 sender 編碼,透過 transport 傳送,在 receiver 解碼。而 "data channels"是直接透過 transport 傳送。

ORTC 最初由 MS Edge 實作支援,但目前 ORTC 很多的物件定義,已經被吸收進入 WebRTC 1.0 核心標準中。

WebRTC, ORTC difference

ref: ORTC draft spec

ref: WebRTC的现状和未来(上)

References

WebRTC API

samples

WebRTC wiki

30-26之 WebRTC 的 P2P 即時通信與小範例