2024/07/22

JAX-RS Jersey in Tomcat

在 java 開發 RESTful Web service 的方式,除了比較常見的 Spring MVC 以外,還有一個隸屬於 JSR-370 規格的 JAX-RS API,這份規格定義了在 java web container 裡面,應該提供什麼 API 介面輔助開發者開發 Representational State Transfer (REST) web service。Eclipse Jersey 則是實作了 JAX-RS API 的一組 library,以下記錄如何在 tomcat 10 裡面使用 Jersey

pom.xml

以下定義了 Maven POM xml,裡面主要引用了 jakarta.ws.rs-api JAX-RS API,以及三個 Jersey 實作的 libary,另外還用了 log4j2

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.maxkit</groupId>
    <artifactId>testweb</artifactId>
    <packaging>war</packaging>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
    </properties>
    <version>1.0</version>
    <name>testweb</name>
    <url>http://maven.apache.org</url>
    <dependencies>
        <!--junit5-->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.10.1</version>
        </dependency>

        <!-- log4j2 + slf4j -->
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.22.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.22.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j-impl</artifactId>
            <version>2.22.1</version>
        </dependency>

        <!-- the REST API -->
        <dependency>
            <groupId>jakarta.ws.rs</groupId>
            <artifactId>jakarta.ws.rs-api</artifactId>
            <version>3.1.0</version>
        </dependency>
        <!-- These next two are needed because we are using Tomcat, which is not a full JEE server - it's just a servlet container. -->

        <!-- the Jersey implementation -->
        <dependency>
            <groupId>org.glassfish.jersey.containers</groupId>
            <artifactId>jersey-container-servlet</artifactId>
            <version>3.1.5</version>
        </dependency>

        <!-- also needed for dependency injection -->
        <dependency>
            <groupId>org.glassfish.jersey.inject</groupId>
            <artifactId>jersey-hk2</artifactId>
            <version>3.1.5</version>
        </dependency>

        <!-- support for using Jackson (JSON) with Jersey -->
        <dependency>
            <groupId>org.glassfish.jersey.media</groupId>
            <artifactId>jersey-media-json-jackson</artifactId>
            <version>3.1.5</version>
        </dependency>

    </dependencies>
    <build>
        <finalName>testweb</finalName>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.12.1</version>
                    <configuration>
                        <source>${maven.compiler.source}</source>
                        <target>${maven.compiler.target}</target>
                        <encoding>${project.build.sourceEncoding}</encoding>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>3.3.1</version>
                    <configuration>
                        <encoding>${project.build.sourceEncoding}</encoding>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

application

有兩種方式定義 application,一種是在 web.xml,一種是直接寫在 Java code 裡面,以 annotation 定義

方法1 web.xml

以下用 org.glassfish.jersey.servlet.ServletContainer 定義了一個 JerseyRestServlet,並對應到 /rest/* 這個 url-pattern。

jersey.config.server.provider.packages 則是這個 application 會使用到的 web service 實作的 java classes 會放在哪一個 java package 裡面

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         id="testweb" version="3.0">
    <display-name>testweb</display-name>

    <servlet>
        <servlet-name>JerseyRestServlet</servlet-name>
        <servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
        <init-param>
            <param-name>jersey.config.server.provider.packages</param-name>
            <param-value>com.maxkit.testweb.jersey.rest</param-value>
        </init-param>
        <init-param>
            <param-name>jersey.config.server.provider.scanning.recursive</param-name>
            <param-value>true</param-value>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>JerseyRestServlet</servlet-name>
        <url-pattern>/rest/*</url-pattern>
    </servlet-mapping>

    <security-constraint>
        <web-resource-collection>
            <web-resource-name>Forbidden</web-resource-name>
            <url-pattern>/WEB-INF/*</url-pattern>
        </web-resource-collection>
        <auth-constraint />
    </security-constraint>
</web-app>

方法2 Annotation

透過 @ApplicationPath 這個 annotation,tomcat 在啟動時,會自動掃描,並定義一個新的 /rest2/* url-pattern 的 application。

這個 application 會使用到的 web service 實作的 java classes 會放在哪一個 java package 裡面,是用 packages() 定義

package com.maxkit.testweb.jersey;

import jakarta.ws.rs.ApplicationPath;
import org.glassfish.jersey.server.ResourceConfig;

@ApplicationPath("rest2")
public class Rest2Application extends ResourceConfig {

    public Rest2Application() {
        packages("com.maxkit.testweb.jersey.rest2");
    }
}

web service

JAX-RS 定義了一些 annotation,標註為 web service

  • @Path

    url method 的相對路徑

  • @GET,@PUT,@POST,@DELETE

    使用哪一個 http method

  • @Consumes

    可接受的 request 裡面資料的 MIME type

  • @Produces

    回傳的資料的 MIME type

  • @PathParam,@QueryParam,@HeaderParam,@CookieParam,@MatrixParam,@FormParam

    參數的來源,@PathParam來自於URL的路徑,@QueryParam來自於URL的查詢參數,@HeaderParam來自於HTTP請求的頭信息,@CookieParam來自於HTTP請求的Cookie

@Path("/demo") 的部分,定義了對應的url,以 ping method 為例,要呼叫 ping,可使用這個 url: http://localhost:8080/testweb/rest/demo/ping

getNotification, postNotification 是測試要再回傳的資料中,以 JSON 格式回傳

login 裡面有用到 Cookie,因為 Jersey 的 Cookie 沒有 maxAge 的功能,這邊加上 maxAge,在 response 透過 Response.ResponseBuilder 設定 cookie

package com.maxkit.testweb.jersey.rest;

import com.maxkit.testweb.jersey.JerseyCookie;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.ws.rs.*;
import jakarta.ws.rs.core.Context;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Date;
import java.util.UUID;

//
// Pingable at:
// http://localhost:8080/testweb/rest/demo/ping
//
@Path("/demo")
public class DemoCommand {
    Logger log = LoggerFactory.getLogger(this.getClass().getName());

    @GET
    @Path("/ping")
    public Response ping() {
        log.debug("ping");
        return Response.ok().entity("Service online").build();
    }

    @GET
    @Path("/get/{id}")
    @Produces(MediaType.APPLICATION_JSON)
    public Response getNotification(@PathParam("id") int id) {
        return Response.ok()
                .entity(new DemoPojo(id, "test message"))
                .build();
    }

    @POST
    @Path("/post")
    @Consumes(MediaType.APPLICATION_JSON)
    @Produces(MediaType.APPLICATION_JSON)
    public Response postNotification(DemoPojo pojo) {
        log.debug("demopojo={}", pojo);
        if( pojo.getTestdate() == null ) {
            pojo.setTestdate(new Date());
        }
        return Response.status(201).entity(pojo).build();
    }

    @POST
    @Path("/login")
    @Produces(MediaType.APPLICATION_JSON)
    public Response login(@Context HttpServletRequest req, @FormParam("userid") String userid, @FormParam("pd") String pwd) {
        JerseyCookie cookie = null;
        String cookiekey = UUID.randomUUID().toString();

        int validtime = 1 * 86400;
        cookie = new JerseyCookie("randomuuid", cookiekey, "/", null, 0, null, validtime, false, false);

        DemoPojo demo = new DemoPojo();
        demo.setId(12345);
        demo.setMessage("message");
        demo.setTestdate(new Date());
        Response.ResponseBuilder builder = Response.ok(demo);
        return builder.header("Set-Cookie", cookie).build();
    }
}

DemoPojo 是單純的一個 java data class

package com.maxkit.testweb.jersey.rest;

import java.util.Date;

public class DemoPojo {

    private int id;
    private String message;
    private Date testdate;

    public DemoPojo() {
    }

    public DemoPojo(int id, String message) {
        this.id = id;
        this.message = message;
    }

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getMessage() {
        return message;
    }

    public void setMessage(String message) {
        this.message = message;
    }

    public Date getTestdate() {
        return testdate;
    }

    public void setTestdate(Date testdate) {
        this.testdate = testdate;
    }
}

JerseyCookie.java

package com.maxkit.testweb.jersey;

import org.glassfish.jersey.message.internal.HttpHeaderReader;
import org.glassfish.jersey.message.internal.StringBuilderUtils;

import jakarta.ws.rs.core.Cookie;
import jakarta.ws.rs.core.NewCookie;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Locale;
import java.util.TimeZone;
//import com.sun.jersey.core.impl.provider.header.WriterUtil;

public class JerseyCookie extends Cookie {
    public static final int DEFAULT_MAX_AGE = -1;

    private String comment = null;
    private int maxAge = DEFAULT_MAX_AGE;
    private boolean secure = false;
    private Date expires;
    private boolean httponly = false;

    public JerseyCookie(String name, String value) {
        super(name, value);
    }

    public JerseyCookie(String name, String value, String path, String domain, String comment, int maxAge,
            boolean secure) {
        super(name, value, path, domain);
        this.comment = comment;
        this.maxAge = maxAge;
        this.secure = secure;
    }

    public JerseyCookie(String name, String value, String path, String domain, int version, String comment, int maxAge,
            boolean secure) {
        super(name, value, path, domain, version);
        this.comment = comment;
        this.maxAge = maxAge;
        this.secure = secure;
    }

    public JerseyCookie(Cookie cookie) {
        super(cookie == null ? null : cookie.getName(), cookie == null ? null : cookie.getValue(),
                cookie == null ? null : cookie.getPath(), cookie == null ? null : cookie.getDomain(),
                cookie == null ? Cookie.DEFAULT_VERSION : cookie.getVersion());
    }

    public JerseyCookie(String name, String value, String path, String domain, int version, String comment, int maxAge,
            boolean secure, boolean httponly) {
        super(name, value, path, domain, version);
        this.comment = comment;
        this.maxAge = maxAge;
        this.secure = secure;
        if (maxAge > 0) {
            expires = new Date(new Date().getTime() + maxAge * 1000);
        }
        this.httponly = httponly;
    }

    public JerseyCookie(Cookie cookie, String comment, int maxAge, boolean secure, boolean httponly) {
        this(cookie);
        this.comment = comment;
        this.maxAge = maxAge;
        this.secure = secure;
        if (maxAge > 0) {
            expires = new Date(System.currentTimeMillis() + maxAge * 1000);
        }
        this.httponly = httponly;
    }

    public JerseyCookie(Cookie cookie, String comment, int maxAge, boolean secure) {
        this(cookie);
        this.comment = comment;
        this.maxAge = maxAge;
        this.secure = secure;
    }

    public static NewCookie valueOf(String value) throws IllegalArgumentException {
        if (value == null)
            throw new IllegalArgumentException("NewCookie is null");

        return HttpHeaderReader.readNewCookie(value);
        // return delegate.fromString(value);
    }

    public String getComment() {
        return comment;
    }

    public int getMaxAge() {
        return maxAge;
    }

    public boolean isSecure() {
        return secure;
    }

    public Cookie toCookie() {
        return new Cookie(this.getName(), this.getValue(), this.getPath(), this.getDomain(), this.getVersion());
    }

    @Override
    public String toString() {
        StringBuilder b = new StringBuilder();

        b.append(getName()).append('=');
        StringBuilderUtils.appendQuotedIfWhitespace(b, getValue());

        b.append(";").append("Version=").append(getVersion());

        if (getComment() != null) {
            b.append(";Comment=");
            StringBuilderUtils.appendQuotedIfWhitespace(b, getComment());
        }
        if (getDomain() != null) {
            b.append(";Domain=");
            StringBuilderUtils.appendQuotedIfWhitespace(b, getDomain());
        }
        SimpleDateFormat COOKIE_EXPIRES_HEADER_FORMAT = new SimpleDateFormat("EEE, d MMM yyyy HH:mm:ss z", Locale.US);
        COOKIE_EXPIRES_HEADER_FORMAT.setTimeZone(TimeZone.getTimeZone("GMT"));
        // String cookieExpire = "expires=" +
        // COOKIE_EXPIRES_HEADER_FORMAT.format(expires);
        if (getExpires() != null) {
            b.append(";Expires=");
            b.append(COOKIE_EXPIRES_HEADER_FORMAT.format(expires));
        }
        if (getPath() != null) {
            b.append(";Path=");
            StringBuilderUtils.appendQuotedIfWhitespace(b, getPath());
        }
        // if (getMaxAge() != -1) {
        // b.append(";Max-Age=");
        // b.append(getMaxAge());
        // }
        if (isSecure())
            b.append(";Secure");
        if (isHttponly())
            b.append(";HTTPOnly");
        return b.toString();
    }

    @Override
    public int hashCode() {
        int hash = super.hashCode();
        hash = 59 * hash + (this.comment != null ? this.comment.hashCode() : 0);
        hash = 59 * hash + this.maxAge;
        hash = 59 * hash + (this.secure ? 1 : 0);
        return hash;
    }


    @Override
    public boolean equals(Object obj) {
        if (obj == null) {
            return false;
        }
        if (getClass() != obj.getClass()) {
            return false;
        }
        final JerseyCookie other = (JerseyCookie) obj;
        if (this.getName() != other.getName() && (this.getName() == null || !this.getName().equals(other.getName()))) {
            return false;
        }
        if (this.getValue() != other.getValue()
                && (this.getValue() == null || !this.getValue().equals(other.getValue()))) {
            return false;
        }
        if (this.getVersion() != other.getVersion()) {
            return false;
        }
        if (this.getPath() != other.getPath() && (this.getPath() == null || !this.getPath().equals(other.getPath()))) {
            return false;
        }
        if (this.getDomain() != other.getDomain()
                && (this.getDomain() == null || !this.getDomain().equals(other.getDomain()))) {
            return false;
        }
        if (this.getExpires() != other.getExpires()
                && (this.getExpires() == null || !this.getExpires().equals(other.getExpires()))) {
            return false;
        }
        if (this.comment != other.comment && (this.comment == null || !this.comment.equals(other.comment))) {
            return false;
        }
        if (this.maxAge != other.maxAge) {
            return false;
        }

        if (this.secure != other.secure) {
            return false;
        }
        if (this.httponly != other.httponly) {
            return false;
        }
        return true;
    }

    public Date getExpires() {
        return expires;
    }

    public boolean isHttponly() {
        return httponly;
    }

}

References

JAX-RS - 維基百科,自由的百科全書

Jakarta REST (JAX-RS) on Tomcat 10 - northCoder

Eclipse Jersey

2024/07/15

CBOR

RFC 8949 CBOR: Concise Binary Object Representation,這是一種輕量的二進位資料格式,可簡單理解為一種二進位的 JSON,設計目的是最小化處理程式碼,最小化訊息位元組大小。另外在 RFC 9052, 9053 CBOR Object Signing and Encryption (COSE) 制定了要如何對 CBOR 資料進行資料簽章與加密的方法。

online test

cbor.me

CBOR/web (plain)

可到這兩個網頁進行線上測試

測試時可以發現,不管是整數,字串,都有對應的編碼。甚至是 JSON 字串,也有對應的 Map (key-value pair) 的編碼方式

CBOR

header 有兩個資訊,前 3 bits 是 Major Type,後 5 個 bits 是 Additional Information

字節(byte)數 1 byte (CBOR Data Item Header) 動態長度 動態長度
結構 Major Type Additional Information Payload Length(可選) Data Payload(可選)
bit 3 Bits 5 Bits 動態長度 動態長度

CBOR 有八種 Major Type

首位元組的 lower 5 bits 在不同的主類型表示長度(除主類型 0 和主類型 1),如果長度指示不足,則依次使用後續位元組。

Major Type Name content
0 unsigned integer -
1 negative integer -
2 byte string N bytes
3 text string N bytes (UTF-8 text)
4 array of data items N data items
5 map of pairs of data items 2N data items (key/value pairs)
6 tag of number N 1 data item
7 simple/float -

比較特別的是 Major Type 6 裡面定義了 date time

0xC0 定義了 text-based date/time,0xC1 定義了 epoch-based date/time

還有 Major Type 7 裡面的

  • False 編碼後 0xF4
  • True 編碼後 0xF5
  • Null 編碼後 0b1110_0000 + 0b0001_01110 (22) = 0xF6

implementation

CBOR — Concise Binary Object Representation | Implementations

這邊列出了不同程式語言的 CBOR 實作 library

這邊用 js 測試

<html>
<script src="https://cdn.jsdelivr.net/npm/cbor-js@0.1.0/cbor.min.js"></script>
<script>

function hex(buffer) {
    var s = '', h = '0123456789ABCDEF';
    (new Uint8Array(buffer)).forEach((v) => { s += h[v >> 4] + h[v & 15]; });
    return s;
}

function test0() {
    console.log("");
    console.log("test0 unsigned integer");
    var initial = 1000;
    var encoded = CBOR.encode(initial);
    var decoded = CBOR.decode(encoded);
    console.log("initial =" + initial);
    console.log("encoded hex=", hex(encoded));
    console.log("decoded =" + decoded);
}

function test1() {
    console.log("");
    console.log("test1 negative integer");
    var initial = -1000;
    var encoded = CBOR.encode(initial);
    var decoded = CBOR.decode(encoded);
    console.log("initial =" + initial);
    console.log("encoded hex=", hex(encoded));
    console.log("decoded =" + decoded);
}

function test2() {
    console.log("");
    console.log("test2 byte string");
    var initial = new Uint8Array([0, 1, 2, 3]);
    var encoded = CBOR.encode(initial);
    var decoded = CBOR.decode(encoded);
    console.log("initial =", initial);
    console.log("encoded hex=", hex(encoded));
    console.log("decoded =", decoded);
}

function test3() {
    console.log("");
    console.log("test3 text string");
    var initial = "text string";
    var encoded = CBOR.encode(initial);
    var decoded = CBOR.decode(encoded);
    console.log("initial =", initial);
    console.log("encoded hex=", hex(encoded));
    console.log("decoded =", decoded);
}

function test4() {
    console.log("");
    console.log("test4 array");
    var initial = [1, "2"];
    var encoded = CBOR.encode(initial);
    var decoded = CBOR.decode(encoded);
    console.log("initial =", initial);
    console.log("encoded hex=", hex(encoded));
    console.log("decoded =", decoded);
}

function test5() {
    console.log("");
    console.log("test5 map");
    var initial = { Hello: "World" };
    var encoded = CBOR.encode(initial);
    var decoded = CBOR.decode(encoded);
    console.log("initial =", initial);
    console.log("encoded hex=", hex(encoded));
    console.log("decoded =", decoded);
}

function test6() {
    console.log("");
    console.log("test6 float");
    var initial = 3.1415;
    var encoded = CBOR.encode(initial);
    var decoded = CBOR.decode(encoded);
    console.log("initial =", initial);
    console.log("encoded hex=", hex(encoded));
    console.log("decoded =", decoded);
}

function test7() {
    console.log("");
    console.log("test7 true");
    var initial = true;
    var encoded = CBOR.encode(initial);
    var decoded = CBOR.decode(encoded);
    console.log("initial =", initial);
    console.log("encoded hex=", hex(encoded));
    console.log("decoded =", decoded);
}

function test8() {
    console.log("");
    console.log("test8 null");
    var initial = null;
    var encoded = CBOR.encode(initial);
    var decoded = CBOR.decode(encoded);
    console.log("initial =", initial);
    console.log("encoded hex=", hex(encoded));
    console.log("decoded =", decoded);
}

test0();
test1();
test2();
test3();
test4();
test5();
test6();
test7();
test8();

</script>
</html>

結果

test0 unsigned integer
initial =1000
encoded hex= 1903E8
decoded =1000

test1 negative integer
initial =-1000
encoded hex= 3903E7
decoded =-1000

test2 byte string
initial = Uint8Array(4) [0, 1, 2, 3, buffer: ArrayBuffer(4), byteLength: 4, byteOffset: 0, length: 4, Symbol(Symbol.toStringTag): 'Uint8Array']
encoded hex= 4400010203
decoded = Uint8Array(4) [0, 1, 2, 3, buffer: ArrayBuffer(5), byteLength: 4, byteOffset: 1, length: 4, Symbol(Symbol.toStringTag): 'Uint8Array']

test3 text string
initial = text string
encoded hex= 6B7465787420737472696E67
decoded = text string

test4 array
initial = (2) [1, '2']
encoded hex= 82016132
decoded = (2) [1, '2']

test5 map
initial = {Hello: 'World'}
encoded hex= A16548656C6C6F65576F726C64
decoded = {Hello: 'World'}

test6 float
initial = 3.1415
encoded hex= FB400921CAC083126F
decoded = 3.1415

test7 true
initial = true
encoded hex= F5
decoded = true

test8 null
initial = null
encoded hex= F6
decoded = null

references

# 物聯網專用資料交換格式 CBOR

cobr.io

CBOR_百度百科

2024/07/01

WebAuthn

WebAuthn 是 Web 與 Authentication 結合的縮寫,也就是 Web Authentication,用在網頁身份認證。這個規格是由 W3C 與 FIDO (Fast IDentity Online) 提出,該正式API 規格,透過瀏覽器的支援,可透過 Public Key Cryptography 機制註冊並認證使用者,取代傳統密碼形式的認證方式。

最傳統的身份認證機制是用密碼,但因為密碼太多了,又因為資安政策需要常常修改密碼,正常人應該已經沒辦法記得很清楚,哪個網站是用哪一個密碼。

WebAuthn 是  FIDO2 framework 的部分,透過 server, browser, authenticator 的搭配,提供一種 passwordless 無密碼的認證方式。

Demo

webauthn.io 是一個單純的網頁 demo。demo 時分兩個步驟,Register 與 Authenticate。

Register 就是註冊帳號,使用者向 browser 要求要註冊一個新帳號,browser 透過 authenticator 的搭配輔助,產生一組 public key,然後完成註冊的步驟。 authenticator 可能是本機的指紋 scanner,也可以是 USB key,或是目前最容易取得的每一個人的手機,並搭配手機的 FaceID 或是指紋 scanner 輔助做身份認證。

註冊完成後,會取得註冊的 id,然後在透過認證 API,進行使用者身份認證。

Javascript demo

註冊

// should generate from server
const challenge = new Uint8Array(32);
window.crypto.getRandomValues(challenge);

const userID = '5d157c55-ea8b-45ca-870a-da877a9c9916';

const utf8EncodeText = new TextEncoder();
const id = utf8EncodeText.encode(userID);

const publicKeyCredentialCreationOptions = {
    challenge,
    rp: {
        name: "Maxkit RP",
        // 這邊只是 demo,沒有放到某個 https 網站測試,如果是網站,就要改成該網站的 domain name
        //id: "maxkit.com.tw",
    },
    user: {
        id,
        name: "test",
        displayName: "Test",
    },
    pubKeyCredParams: [{alg: -7, type: "public-key"}],
    authenticatorSelection: {
        authenticatorAttachment: "cross-platform",
    },
    timeout: 60000,
    attestation: "direct"
};

const credential = await navigator.credentials.create({
    publicKey: publicKeyCredentialCreationOptions
});

把上面這一部分的 js code 直接貼到 chrome console 執行。主要就是透過 navigator.credentials.create 註冊產生 public key

challenge 欄位是由 server 產生,最終會由 authenticator 以私鑰簽章後,在 credential response 裡面傳回來,讓 server 能夠檢查確認 authenticator

rp.id 應該要填網域 domain name,這邊只是本機測試,不填寫

authenticatorAttachment: "cross-platform" 的意思是,可搭配使用外部認證工具,例如在 NB 瀏覽認證網頁時,可使用手機作為認證工具進行認證。

在 NB 提示認證時,會看到這個畫面

這時候,就用自己的手機相機,拍 QR code,手機上會出現以下畫面

確認後,會在手機的密碼部分,儲存一個專屬的密碼。iOS 手機選單在 "設定 -> 密碼"

在 console 列印 credential 的內容

// print credential
console.log(credential)
PublicKeyCredential {rawId: ArrayBuffer(20), response: AuthenticatorAttestationResponse, authenticatorAttachment: 'cross-platform', id: 'ku22CGGrZdYlkr9cCXL4IWtyYLc', type: 'public-key'}

正常狀況應該把 credential 傳送到 server,檢查並 decode 得到使用者的 id, public key。這邊僅是測試 demo,不做處理。

response.attestationObject 裡面有 public key 跟其他 metadata,是用 CBOR Concise Binary Object Representation 編碼後的 binary data。

認證

以下是認證的 demo code,從剛剛的 credential.id 欄位取得 id,通常應該是儲存在 server,從 server 傳到網頁得到這個資料

const credentialIdStr = credential.id;
const credentialId = Uint8Array.from(window.atob(credentialIdStr), c=>c.charCodeAt(0));

const challenge2 = new Uint8Array(32);
window.crypto.getRandomValues(challenge2);

const publicKeyCredentialRequestOptions = {
    challenge: challenge2,
    //rpId: "maxkit.com.tw",
    allowCredentials: [{
        id: credentialId, // from registration
        type: 'public-key',
        transports: ['usb', 'ble', 'nfc', 'hybrid'],
    }],
    timeout: 60000,
}

const assertion = await navigator.credentials.get({
    publicKey: publicKeyCredentialRequestOptions
});

transports 是可以認證的方式

網頁同樣會出現 QR code,等待 authenticator,這邊我們用剛剛註冊的手機掃描

手機掃描時,會出現一個提示

點"使用通行密鑰登入"

繼續透過 FaceID 認證,認證完成,同樣可取得 response

console.log(assertion)

CTAP

Client-to-Authenticator Protocol,這是在瀏覽器打開註冊/認證網頁後,authnticator 跟瀏覽器之間傳輸的協定,也就是 WebAuthn API 跟外部硬體認證機制溝通的規格

References

WebAuthn guide

什么是WebAuthn:在Web上使用Touch ID和Windows Hello登录_webauthn 网站登录-CSDN博客

你可能不知道的WebAuthN(FIDO) | 又LAG隨性筆記

初探 WebAuthn UAF 的無密碼 Passwordless 登入技術 – 要改的地方太多了,那就改天吧

一起來了解 Web Authentication

CredentialsContainer - Web APIs | MDN

2024/06/17

gradio

gradio 是一組 python library,可快速建立網頁介面的 API,目的是提供給 Machine Learning Model demo 的使用者操作介面。大部分的 AI model 是以 python 開發,且使用程式或文字介面操作,gradio 可快速將使用介面轉換為網頁應用程式,不需要撰寫網頁 css, javascript 等程式碼。

Quickstart 是 gradio 的快速入門說明

安裝 gradio

pip3 install gradio

在執行時,遇到這樣的錯誤訊息

ImportError: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'OpenSSL 1.0.2k-fips  26 Jan 2017'. See: https://github.com/urllib3/urllib3/issues/2168

根據 python - ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3 - Stack Overflow 的說明,要 downgrade urllib3

pip3 uninstall urllib3
pip3 install 'urllib3<2.0'

test1.py,launch 時加上 server_name="0.0.0.0" 參數,可讓機器所有 network interfaces 在 TCP Port 7860 提供網頁服務

import gradio as gr

def greet(name, intensity):
    return "Hello " * intensity + name + "!"

demo = gr.Interface(
    fn=greet,
    inputs=["text", "slider"],
    outputs=["text"],
)

demo.launch(server_name="0.0.0.0")

在瀏覽器連接 http://localhost:7860

如果在 launch 加上參數 share=True

啟動時,會出現這樣的訊息

$ python3 test1.py
Running on local URL:  http://127.0.0.1:7860
Running on public URL: https://424222ce32aa52ca72.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)

這表示可透過一個 public redirect 的網址,連接到這個 service

References

gradio

Creating A Chatbot Fast

2024/06/03

log4j2

java 開發環境,JDK 預設有提供 logging 機制,但 logging 機制的發展,也是一段精彩的歷史。就以往使用的經驗,從一開始的 log4j,後來換成了 logback + slf4j,到現在這個階段,似乎又要換成 log4j2,因為 log4j2 的效能評估,結果已經是又超過了 logback。

logging framwork 的發展可以參考這一篇文章: Java Logging 的歷史與戰爭

但這種情況也說明了,開放且多元的 java 開發環境,一個必要且熱門的 framework,會吸引大家注意並不斷提出更好的框架,但使用者就得跟著時代不斷的進步。

使用 log4j 用 maven 引用 library,在 pom.xml 的 dependencies 裡面加上 log4j2 及 slf4j。

slf4j 是 logging 的 wrapper,可用一致 API 介面進行程式開發,在不修改程式的狀況下,可替換內部的實作為 logback 或 log4j 或 jdk logging。

    <dependencies>
        <!-- log4j2 + slf4j -->
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.22.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.22.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j-impl</artifactId>
            <version>2.22.1</version>
        </dependency>
    </dependencies>

在 classpath 裡面加上設定檔 log4j2.xml

以下是調整後的設定

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout>
                <Pattern>%d{HH:mm:ss.SSS} [%thread] %-5p %c %L - %m%n</Pattern>
            </PatternLayout>
        </Console>

        <!--
        <File name="FILE" fileName="testweb.log">
            <PatternLayout>
                <Pattern>%d [%thread] %-5p %c %L - %m%n</Pattern>
            </PatternLayout>
        </File>
        -->
        <RollingFile name="RollingFile" fileName="logs/testweb.log" filePattern="logs/testweb-%d{yyyy-MM-dd}-%i.log.gz">
            <PatternLayout>
                <Pattern>%d [%thread] %-5p %c %L - %m%n</Pattern>
            </PatternLayout>
            <Policies>
                <OnStartupTriggeringPolicy />
                <TimeBasedTriggeringPolicy />
                <SizeBasedTriggeringPolicy size="250 MB"/>
            </Policies>
            <DefaultRolloverStrategy>
                <Delete basePath="${baseDir}" maxDepth="2">
                    <IfFileName glob="*/test-*.log.gz" />
                    <IfLastModified age="30d" />
                </Delete>
            </DefaultRolloverStrategy>
        </RollingFile>
    </Appenders>
    <Loggers>
        <!-- hibernate -->
        <logger name="org.hibernate" level="INFO" />
        <Logger name="org.hibernate.SQL" level="DEBUG" />
        <logger name="org.hibernate.cfg" level="ERROR" />

        <Root level="info">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="RollingFile"/>
        </Root>
    </Loggers>
</Configuration>

RollingFile 的部分可以參考官方說明Log4j 2 Appenders

Policies 的部分,決定什麼時候要 trigger rolling 的機制。OnStartupTriggeringPolicy 代表程式啟動要做一次。TimeBasedTriggeringPolicy 則是跟著 Pattern 裡面日期的最小精度的部分 trigger,目前日期是設定為日,所以這個範例,每天會處理一次。SizeBasedTriggeringPolicy 則是會判斷 log file 的檔案大小 trigger,pattern 部分會因為 %i 這個設定,每次會將檔案改為 -1, -2 這樣的檔名。

DefaultRolloverStrategy 可將超過 30 天,符合glob 設定的 pattern 的檔案刪除。

使用 logging 是透過 sl4j 介面

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class HelloWorld {
    Logger log = LoggerFactory.getLogger(this.getClass().getName());
    public void sayHello(){
        log.debug("This will be printed on debug");
        log.info("This will be printed on info");
        log.warn("This will be printed on warn");
        log.error("This will be printed on error");

        log.info("Appending string: {}.", "Hello, World");
    }
}

log4j2 對效能最大的改進點在非同步的 file appender Log4j 2 Appenders

調整 log4j2.xml 設定,基於上面的 RollingFile 增加 Appenders。

bufferSize 預設為 1024,代表可暫存 1024 則 log,必須要設定為 2 的指數。

        <Async name="AsyncRollingFile" bufferSize="262144">
            <AppenderRef ref="RollingFile"/>
        </Async>

然後修改下面的 AppenderRef,就可以換成 AsyncRollingFile

        <Root level="info">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="AsyncRollingFile"/>
        </Root>

References

Log4j2中RollingFile的文件滚动更新机制 - Ye_yang - 博客园

Java Logging Frameworks: log4j vs logback vs log4j2

Log4j; Performance

2024/05/27

Guava Reflection

TypeToken

因為 java 有 type erasure 特性,無法在 runtime 得知 generic Class 物件

TypeToken 使用了 workaround 方法,可處理 generict types,就是能夠在 runtime 檢測 Generic Type

    @Test
    public void typeToken() {
        ArrayList<String> stringList = Lists.newArrayList();
        ArrayList<Integer> intList = Lists.newArrayList();
        System.out.println(stringList.getClass().isAssignableFrom(intList.getClass()));
        // returns true, even though ArrayList<String> is not assignable from ArrayList<Integer>

        TypeToken<List<String>> stringListToken = new TypeToken<List<String>>() {};
        TypeToken<List<Integer>> integerListToken = new TypeToken<List<Integer>>() {};
        TypeToken<List<? extends Number>> numberTypeToken = new TypeToken<List<? extends Number>>() {};
        assertFalse(stringListToken.isSubtypeOf(integerListToken));
        assertFalse(numberTypeToken.isSubtypeOf(integerListToken));
        assertTrue(integerListToken.isSubtypeOf(numberTypeToken));
    }

利用 TypeToken 在 runtime 取得 complex type 資訊

    abstract class ParametrizedClass<T> {
        TypeToken<T> type = new TypeToken<T>(getClass()) {};
    }
    @Test public void typeToken2() throws NoSuchMethodException {
        //// 產生一個 class 裡面儲存了 TypeToken 欄位
        ParametrizedClass<String> parametrizedClass = new ParametrizedClass<String>() {};
        assertEquals(parametrizedClass.type, TypeToken.of(String.class));

        //// 也能產生 TypeToken of a complex type,裡面有多個 generic type
        // runtime 還是能取得 type 資訊
        TypeToken<Function<Integer, String>> funToken
                = new TypeToken<Function<Integer, String>>() {};
        TypeToken<?> funResultToken = funToken
                .resolveType(Function.class.getTypeParameters()[1]);
        assertEquals(funResultToken, TypeToken.of(String.class));

        /// 也能在 Map 裡面得知 entry 的 data type
        TypeToken<Map<String, Integer>> mapToken
                = new TypeToken<Map<String, Integer>>() {};
        TypeToken<?> entrySetToken = mapToken
                .resolveType(Map.class.getMethod("entrySet")
                        .getGenericReturnType());
        assertEquals(
                entrySetToken,
                new TypeToken<Set<Map.Entry<String, Integer>>>() {});
    }

Invokable

Invokable 是 java.lang.reflec.Method 及 java.lang.reflect.Constructor 的 fluent wrapper,提供簡化後的 API

    @Test
    public void invokable() throws NoSuchMethodException {
        Method method = CustomClass.class.getMethod("somePublicMethod");
        Invokable<CustomClass, ?> invokable
                = new TypeToken<CustomClass>() {}
                .method(method);
        // 分別透過 Guava 及 標準的 JAVA reflection API 檢測是否有 somePublicMethod 這個 method
        boolean isPublicStandradJava = Modifier.isPublic(method.getModifiers());
        boolean isPublicGuava = invokable.isPublic();

        assertTrue(isPublicStandradJava);
        assertTrue(isPublicGuava);

        ////////
        // checking if a method is overridable
        Method method2 = CustomClass.class.getMethod("notOverridablePublicMethod");
        // 用 Guava TypeToken 比較簡單
        Invokable<CustomClass, ?> invokable2
                = new TypeToken<CustomClass>() {}.method(method2);

        // 用標準 java 的方法
        boolean isOverridableStandardJava = (!(Modifier.isFinal(method2.getModifiers())
                || Modifier.isPrivate(method2.getModifiers())
                || Modifier.isStatic(method2.getModifiers())
                || Modifier.isFinal(method2.getDeclaringClass().getModifiers())));
        boolean isOverridableFinalGauava = invokable2.isOverridable();

        assertFalse(isOverridableStandardJava);
        assertFalse(isOverridableFinalGauava);
    }

References

Guide to Guava's Reflection Utilities | Baeldung

ReflectionExplained · google/guava Wiki · GitHub

2024/05/20

Guava Math

Guava 針對 int, long, BigIntegers, double 提供一些 Math Utility Class methods

IntMath

    @Test
    public void intMath() {
        // binomial(int n, int k)
        // binomial coefficient of n and k
        // 計算 n/k(n-k)
        int result = IntMath.binomial(6, 3);
        assertEquals(20, result);
        // 超過 Integer range 時,會回傳 MAX_VALUE
        result = IntMath.binomial(Integer.MAX_VALUE, 3);
        assertEquals(Integer.MAX_VALUE, result);

        //// ceilingPowerOfTwo(int x)
        // the smallest power of two which is greater than or equal to x
        // x 落在 2 的哪一個指數結果的區間
        // 2^(n-1) < x < 2 ^n
        result = IntMath.ceilingPowerOfTwo(20);
        assertEquals(32, result);

        //// checkedAdd(int a, int b)
        // a+b ,同時會檢查是否結果有 overflow,發生時會 throw ArithmeticException
        result = IntMath.checkedAdd(1, 2);
        assertEquals(3, result);

        Exception exception = assertThrows(ArithmeticException.class, () -> {
            IntMath.checkedAdd(Integer.MAX_VALUE, 100);
        });

        /// divide(int p, int q, RoundingMode mode)
        // 定義 rounding mode 的除法
        result = IntMath.divide(10, 3, RoundingMode.CEILING);
        assertEquals(4, result);
        Exception exception2 = assertThrows(ArithmeticException.class, () -> {
            // 未定義 Rounding Mode  但實際上結果需要 rounding
            IntMath.divide(10, 3, RoundingMode.UNNECESSARY);
        });

        /// factorial(int n)
        // n!   n 階層
        result = IntMath.factorial(5);
        assertEquals(120, result);
        result = IntMath.factorial(Integer.MAX_VALUE);
        assertEquals(Integer.MAX_VALUE, result);

        //// floorPowerOfTwo(int x)
        // 2^n < x < 2 ^(n+1)
        result = IntMath.floorPowerOfTwo(30);
        assertEquals(16, result);

        //// gcd(int a, int b)
        // 最大公因數
        result = IntMath.gcd(30, 40);
        assertEquals(10, result);

        //// isPowerOfTwo(int x)
        // 是否為 2 的倍數
        assertTrue( IntMath.isPowerOfTwo(16) );
        assertFalse( IntMath.isPowerOfTwo(20) );

        //// isPrime(int n)
        // 是否為質數
        assertFalse( IntMath.isPrime(20) );

        //// log10(int x, RoundingMode mode)
        // 以 10 為底 的對數
        result = IntMath.log10(30, RoundingMode.CEILING);
        assertEquals(2, result);
        Exception exception3 = assertThrows(ArithmeticException.class, () -> {
            // 未定義 Rounding Mode  但實際上結果需要 rounding
            IntMath.log10(30, RoundingMode.UNNECESSARY);
        });

        //// log2(int x, RoundingMode mode)
        // 以 2 為底 的對數
        result = IntMath.log2(30, RoundingMode.CEILING);
        assertEquals(5, result);
        Exception exception4 = assertThrows(ArithmeticException.class, () -> {
            // 未定義 Rounding Mode  但實際上結果需要 rounding
            IntMath.log2(30, RoundingMode.UNNECESSARY);
        });

        //// mean(int x, int y)
        //平均
        result = IntMath.mean(30, 20);
        assertEquals(25, result);

        // mod(int x, int m)
        // 餘數
        result = IntMath.mod(30, 4);
        assertEquals(2, result);

        // pow(int b, int k)
        // 指數
        result = IntMath.pow(6, 4);
        assertEquals(1296, result);

        //// saturatedAdd(int a, int b)
        // 控制 overflow/ underflow 時,會改回傳 MAX_VALUE/MIN_VALUE
        result = IntMath.saturatedAdd(6, 4);
        assertEquals(10, result);
        result = IntMath.saturatedAdd(Integer.MAX_VALUE, 1000);
        assertEquals(Integer.MAX_VALUE, result);

        //// sqrt(int x, RoundingMode mode)
        // 平方根
        result = IntMath.sqrt(30, RoundingMode.CEILING);
        assertEquals(6, result);
        Exception exception5 = assertThrows(ArithmeticException.class, () -> {
            // 未定義 Rounding Mode  但實際上結果需要 rounding
            IntMath.sqrt(30, RoundingMode.UNNECESSARY);
        });

    }

LongMath

大部分跟 IntMath 一樣,除了 mod

    @Test
    public void longMath() {
        // Most operations are similar to the IntMath utility
        // 以下是不同的地方

        /// mod(long x, int m) and mod(long x, long m)
        // x mod m
        int result = LongMath.mod(30L, 4);
        assertEquals(2, result);
        long result2 = LongMath.mod(30L, 4L);
        assertEquals(2L, result);
    }

BigIntegerMath

跟 IntMath 類似

DoubleMath

    @Test
    public void doubleMath() {
        //// isMathematicalInteger(double x)
        // 是否為整數
        boolean result = DoubleMath.isMathematicalInteger(5);
        assertTrue(result);
        result = DoubleMath.isMathematicalInteger(5.2);
        assertFalse(result);


        //// log2(double x)
        double result2 = DoubleMath.log2(4);
        assertEquals(2, result2, 0);
    }

References

MathExplained · google/guava Wiki · GitHub

Guide to Mathematical Utilities in Guava | Baeldung

2024/05/13

Guava Hashing

Guava Hash 裡面有幾個重要的類別:Hashing, HashFunction, Hasher, HashCode, Funnel, PrimitiveSink

HashFunction

HashFunction 是 stateless function,可將任意的 data 轉換為 fixed number of bits,相同的 input 會得到相同的 output 結果,不同的 input 得到不同的唯一的結果

可產生 Hasher 物件,也可以根據條件回傳 HashCode 結果

    @Test
    public void hashFunction() {
        HashFunction hf = Hashing.md5();
        HashCode hc = hf.newHasher()
                .putLong(10)
                .putString("abc", Charsets.UTF_8)
                .hash();
        System.out.println("hashFunction="+hc.toString());
    }

Hasher

HashFunction 可以使用 stateful Hasher,Hasher 提供 fluent syntax,可將資料填充到 Hash 裡面,並得到 hash value。Hasher 可接受 primitive input, byte array, slices of byte arrays, character sequences, 任意物件產生的 Funnel。

資料透過 putXXX() method 放到 Hasher 的資料源中,有 putByte, putBytes, putShort, putInt, putLong, putFloat, putdouble, putBoolean, putChar, putEnencodedChars, putString, putObject

Funnel

如果有一個類別

class Person {
  final int id;
  final String firstName;
  final String lastName;
  final int birthYear;
}

其 Funnel 會是

Funnel<Person> personFunnel = new Funnel<Person>() {
  @Override
  public void funnel(Person person, PrimitiveSink into) {
    into
        .putInt(person.id)
        .putString(person.firstName, Charsets.UTF_8)
        .putString(person.lastName, Charsets.UTF_8)
        .putInt(birthYear);
  }
};

組合起來的code

    @Test
    public void objectToHashCode() {

        // 需要hash的对象
        Person person = new Person(27, "John", "Wink", 1990);

        Funnel<Person> personFunnel = new Funnel<Person>() {
            @Override
            public void funnel(Person person, PrimitiveSink into) {
                into
                        .putInt(person.id)
                        .putString(person.firstName, Charsets.UTF_8)
                        .putString(person.lastName, Charsets.UTF_8)
                        .putInt(person.birthYear);
            }
        };

        // md5算法
        HashFunction hf = Hashing.md5();
        HashCode hc = hf.newHasher()
                .putString("abc", Charsets.UTF_8)
                .putObject(person, personFunnel)
                .hash();
        System.out.println("objectToHashCode="+hc.toString());
    }

    class Person {
        int id;
        String firstName;
        String lastName;
        int birthYear;

        public Person(int id, String firstName, String lastName, int birthYear) {
            this.id = id;
            this.firstName = firstName;
            this.lastName = lastName;
            this.birthYear = birthYear;
        }
    }

HashCode

當 Hasher 給了所有 input 資料,他的 hash() method 會產生 HashCode 結果

HashCode 支援 euqality testing 例如 asInt(), asLong, asBytes()

另外有 writeBytesTo(array, offset, maxLength) 可將 hash 的 maxLength bytes 寫入 array

BloomFilter

可偵測物件是否 "definitely" 不在 filter 裡面,或是有可能已經被加入 Bloom filter

對於 hash 來說,就是偵測是否 hash value 有碰撞,不同的 input 得到相同的結果

BloomFilter<Person> friends = BloomFilter.create(personFunnel, 500, 0.01);
for (Person friend : friendsList) {
  friends.put(friend);
}
// much later
if (friends.mightContain(someone)) {
  // the probability that someone reached this place if they aren't a friend is
  // 1% we might, for example, start asynchronously loading things for someone
  // while we do a more expensive exact check
}

Hashing

Hashing (Guava: Google Core Libraries for Java HEAD-jre-SNAPSHOT API)

裡面有 static method,可取得某些演算法的 HashFunction

  • adler32()

  • crc32(), crc32c()

  • farmHashFingerprint64()

  • fingerprint2011()

  • goodFastHash(int minimumBits)

  • hmacMd5

  • hmacSha1

  • hmacSha256

  • hmacSha512

  • md5()

  • murmur3_128(), murmur3_32()

  • sha1(), sha256(), sha384(), sha512()

  • sipHash24

    @Test
    public void hashing() {
        // 測試 Hasing 演算法
        String input = "hello, world";
        // MD5
        System.out.println("md5="+Hashing.md5().hashBytes(input.getBytes()).toString());
        // sha256
        System.out.println("sha256="+Hashing.sha256().hashBytes(input.getBytes()).toString());
        // sha512
        System.out.println("sha512="+Hashing.sha512().hashBytes(input.getBytes()).toString());
        // crc32
        System.out.println("crc32="+Hashing.crc32().hashBytes(input.getBytes()).toString());
        // MD5
        System.out.println("MD5="+Hashing.md5().hashUnencodedChars(input).toString());
    }

References

HashingExplained · google/guava Wiki · GitHub

2024/05/06

Guava String Utilities 2

CharMatcher

CharMatcher 用在字元的 trimming, collapsing, removing, retaining

    @Test
    public void char_matchers() {
        String string = "12 ab 34 CD\r\n 56 ef789GH";
        String noControl = CharMatcher.javaIsoControl().removeFrom(string); // remove control characters
        String theDigits = CharMatcher.digit().retainFrom(string); // only the digits
        String spaced = CharMatcher.whitespace().trimAndCollapseFrom(string, ' ');
        // trim whitespace at ends, and replace/collapse whitespace into single spaces
        String noDigits = CharMatcher.javaDigit().replaceFrom(string, "*"); // star out all digits
        String lowerAndDigit = CharMatcher.javaDigit().or(CharMatcher.javaLowerCase()).retainFrom(string);
        // eliminate all characters that aren't digits or lowercase

        assertEquals("12 ab 34 CD 56 ef789GH", noControl);
        assertEquals("123456789", theDigits);
        assertEquals("12 ab 34 CD 56 ef789GH", spaced);
        assertEquals("** ab ** CD\r\n ** ef***GH", noDigits);
        assertEquals("12ab3456ef789", lowerAndDigit);
    }

除了一些已經封裝的methods 以外,通用的 method 有三個

Method Description example
anyOf(CharSequence) 要符合的字元 CharMatcher.anyOf("aeiou")
is(char) 特定的 char
inRange(char, char) 一連串的字元 CharMatcher.inRange('a', 'z')
    @Test
    public void char_matchers2() {
        String string = "12 ab 34 CD\r\n 56";
        String anyOfResult = CharMatcher.anyOf("cdef\r\n").removeFrom(string);
        String isResult = CharMatcher.is('C').removeFrom(string);
        String inRangeResult = CharMatcher.inRange('A', 'Z').removeFrom(string);

        assertEquals("12 ab 34 CD 56", anyOfResult);
        assertEquals("12 ab 34 D\r\n 56", isResult);
        assertEquals("12 ab 34 \r\n 56", inRangeResult);
    }

使用 CharMatcher 的 methods

Method Description
collapseFrom(CharSequence, char) 將一連串的字元,縮小變成一個  ex: WHITESPACE.collapseFrom(string, ' ') 會減少為只有一個空白字元
matchesAllOf(CharSequence) 符合所有字元
removeFrom(CharSequence) 移除符合字元
retainFrom(CharSequence) 保留符合的字元
trimFrom(CharSequence) 去掉 leading, trailing 符合的字元
replaceFrom(CharSequence, CharSequence) 取代字元

Charsets

注意 "不要" 這樣寫

try {
  bytes = string.getBytes("UTF-8");
} catch (UnsupportedEncodingException e) {
  // how can this possibly happen?
  throw new AssertionError(e);
}

要改用 Charsets

bytes = string.getBytes(Charsets.UTF_8);

CaseFormat

Format Example
LOWER_CAMEL lowerCamel
LOWER_HYPHEN lower-hyphen
LOWER_UNDERSCORE lower_underscore
UPPER_CAMEL UpperCamel
UPPER_UNDERSCORE UPPER_UNDERSCORE
CaseFormat.UPPER_UNDERSCORE.to(CaseFormat.LOWER_CAMEL, "CONSTANT_NAME"));
// returns "constantName"

References

StringsExplained · google/guava Wiki · GitHub

2024/04/29

Guava String Utilities 1

Joiner

可連接多個 string,同時避免遇到 null 會發生問題

joiner configuration methods 永遠會回傳新的 Joiner 物件,因此 Joiner 是 thread safe 的

    @Test
    public void joiner() {
        Joiner joiner = Joiner.on("; ").skipNulls();
        String joinresult = joiner.join("Harry", null, "Ron", "Hermione");

        assertEquals("Harry; Ron; Hermione", joinresult);

        Joiner joiner2 = Joiner.on("; ").useForNull("?");
        String joinresult2 = joiner2.join("Harry", null, "Ron", "Hermione");
        assertEquals("Harry; ?; Ron; Hermione", joinresult2);

        String joinresult3 = Joiner.on(",").join(Arrays.asList(1, 5, 7));
        assertEquals("1,5,7", joinresult3);
    }

Splitter

原本的 String.split 有些奇怪的回傳結果,Splitter提供多個 method,可以自己決定回傳的結果內容。另外 Splitter 可以套用 Pattern, char, Stirng 或 CharMatcher 作為分割標準

Factory method

Method Description Example
Splitter.on(char) 以字元分割 Splitter.on(';')
Splitter.on(CharMatcher) 以一組字元分割 Splitter.on(CharMatcher.BREAKING_WHITESPACE)
Splitter.on(CharMatcher.anyOf(";,."))
Splitter.on(String) 以字串分割 Splitter.on(", ")
Splitter.on(Pattern) 以 regular expression 分割 Splitter.onPattern("\r?\n")
Splitter.onPattern(String)
Splitter.fixedLength(int) 固定長度分割 Splitter.fixedLength(3)

Modifier methods

Method Description Example Result
omitEmptyString() 忽略 empty string Splitter.on(',').omitEmptyStrings().split("a,,c,d") "a", "c", "d"
trimResults() 去掉 whitespace Splitter.on(',').trimResults().split("a, b, c, d") "a", "b", "c", "d"
trimResults(CharMatcher) 去掉一組字元(最前面&最後面) Splitter.on(',').trimResults(CharMatcher.is('')).split("_a ,_b ,c__") "a ", "b_ ", "c"
limit(int) 最多只分割為 int 個字串後,就不再分割 Splitter.on(',').limit(3).split("a,b,c,d") "a", "b", "c,d"

從 split() 改用 splitToList 會回傳 List

@Test
    public void splitter() {
        String[] split1 = ",a,,b,".split(",");
        // 原生 split 會自動去掉最後一個 ""
        // "", "a", "", "b"
        assertEquals("[, a, , b]", Arrays.toString(split1));

        Iterable<String> split2 =
                Splitter.on(',')
                .trimResults()
                .omitEmptyStrings()
                .split("foo,bar,,   qux");
        // "foo", "bar", "qux"
        assertEquals("[foo, bar, qux]", split2.toString());

        Iterable<String> split3 = Splitter.on(',').omitEmptyStrings().split("a,,c,d");
        // "a", "c", "d"
        assertEquals("[a, c, d]", split3.toString());

        Iterable<String> split4 = Splitter.on(',').trimResults().split("a, b, c, d");
        // "a", "b", "c", "d"
        assertEquals("[a, b, c, d]", split4.toString());

        Iterable<String> split5 = Splitter.on(',').trimResults(CharMatcher.is('_')).split("_a ,_b_ ,c__");
        //"a ", "b_ ", "c"
        assertEquals("[a , b_ , c]", split5.toString());

        Iterable<String> split6 = Splitter.on(',').limit(3).split("a,b,c,d");
        //"a", "b", "c,d"
        assertEquals("[a, b, c,d]", split6.toStringa
());


        // 改用 splitToList 會回傳 List<String>
        List<String> split7 = Splitter.on(',').omitEmptyStrings().splitToList("a,,c,d");
    }

How to use Joiner Splitter

Convert List into String Using Joiner

    @Test
    public void whenConvertListToString_thenConverted() {
        List<String> names = Lists.newArrayList("Taipei", "Taichung", "Kaohsiung");
        String result = Joiner.on(",").join(names);

        assertEquals(result, "Taipei,Taichung,Kaohsiung");
    }

Convert Map to String Using Joiner

    @Test
    public void whenConvertMapToString_thenConverted() {
        Map<String, Integer> salary = Maps.newHashMap();
        salary.put("Taipei", 1000);
        salary.put("Taichung", 1500);
        String result = Joiner.on(" , ").withKeyValueSeparator(" = ")
                .join(salary);

        assertTrue(result.contains("Taipei = 1000"));
        assertTrue(result.contains("Taichung = 1500"));
    }

Join Nested Collections

    @Test
    public void whenJoinNestedCollections_thenJoined() {
        List<ArrayList<String>> nested = Lists.newArrayList(
                Lists.newArrayList("apple", "banana", "orange"),
                Lists.newArrayList("cat", "dog", "bird"),
                Lists.newArrayList("John", "Jane", "Adam"));
        String result = Joiner.on(";").join(Iterables.transform(nested,
                new com.google.common.base.Function<List<String>, String>() {
                    @Override
                    public String apply(List<String> input) {
                        return Joiner.on("-").join(input);
                    }
                }));

        assertTrue(result.contains("apple-banana-orange"));
        assertTrue(result.contains("cat-dog-bird"));
        assertTrue(result.contains("John-Jane-Adam"));
    }

Handle Null Values While Using Joiner

    @Test
    public void whenConvertListToStringAndSkipNull_thenConverted() {
        List<String> names = Lists.newArrayList("John", null, "Jane", "Adam", "Tom");
        String result = Joiner.on(",").skipNulls().join(names);

        assertEquals(result, "John,Jane,Adam,Tom");
    }

    @Test
    public void whenUseForNull_thenUsed() {
        List<String> names = Lists.newArrayList("John", null, "Jane", "Adam", "Tom");
        String result = Joiner.on(",").useForNull("nameless").join(names);

        assertEquals(result, "John,nameless,Jane,Adam,Tom");
    }

Create Map From String Using Splitter

    @Test
    public void whenCreateMapFromString_thenCreated() {
        String input = "John=first,Adam=second";
        Map<String, String> result = Splitter.on(",")
                .withKeyValueSeparator("=")
                .split(input);

        assertEquals("first", result.get("John"));
        assertEquals("second", result.get("Adam"));
    }

Split String With Multiple Separators

    @Test
    public void whenSplitStringOnMultipleSeparator_thenSplit() {
        String input = "apple.banana,,orange,,.";
        List<String> result = Splitter.onPattern("[.,]")
                .omitEmptyStrings()
                .splitToList(input);

        assertTrue(result.contains("apple"));
        assertTrue(result.contains("banana"));
        assertTrue(result.contains("orange"));
    }

Split a String at Specific Length

    @Test
    public void whenSplitStringOnSpecificLength_thenSplit() {
        String input = "Hello world";
        List<String> result = Splitter.fixedLength(3).splitToList(input);
        //Hel", "lo ", "wor", "ld"
        assertEquals("[Hel, lo , wor, ld]", result.toString());
    }

Limit the Split Result

    @Test
    public void whenLimitSplitting_thenLimited() {
        String input = "a,b,c,d,e";
        List<String> result = Splitter.on(",")
                .limit(4)
                .splitToList(input);

        assertEquals(4, result.size());
        // ["a", "b", "c", "d,e"]
        assertEquals("[a, b, c, d,e]", result.toString());
    }

References

StringsExplained · google/guava Wiki · GitHub

Guava - Join and Split Collections | Baeldung

2024/04/22

Guava Cache

Guava Cache 提供了基本操作,eviction policies,refresh cache 這些功能

How to Use

    @Test
    public void whenCacheMiss_thenValueIsComputed() {
        // caching the uppercase form of String instances
        CacheLoader<String, String> loader;
        loader = new CacheLoader<String, String>() {
            @Override
            public String load(String key) {
                return key.toUpperCase();
            }
        };

        LoadingCache<String, String> cache;
        cache = CacheBuilder.newBuilder().build(loader);

        // getUnchecked,會透過 CacheLoader 的 load 載入 cache 裡面,並取得 cache 的結果
        assertEquals(0, cache.size());
        assertEquals("HELLO", cache.getUnchecked("hello"));
        assertEquals(1, cache.size());
        assertEquals("HELLO", cache.getUnchecked("Hello"));
        assertEquals(2, cache.size());
    }

Eviction Policies

  • cache size

  • cache size with weight function

  • idle time

  • total live time

    @Test
    public void whenCacheReachMaxSize_thenEviction() {
        // 設定 cache 的 maximumSize(3)
        CacheLoader<String, String> loader = createLoader();
        LoadingCache<String, String> cache;
        cache = CacheBuilder.newBuilder().maximumSize(3).build(loader);

        cache.getUnchecked("first");
        cache.getUnchecked("second");
        cache.getUnchecked("third");
        // 放入第四個,會將第一個 cached 資料刪除
        cache.getUnchecked("forth");
        assertEquals(3, cache.size());
        assertNull(cache.getIfPresent("first"));
        assertEquals("FORTH", cache.getIfPresent("forth"));
    }

    @Test
    public void whenCacheReachMaxWeight_thenEviction() {
        CacheLoader<String, String> loader = createLoader();

        // 定義 Weigher 的 weight function
        // 以 cached object 的 value 的字串長度,作為 weight
        Weigher<String, String> weighByLength;
        weighByLength = new Weigher<String, String>() {
            @Override
            public int weigh(String key, String value) {
                return value.length();
            }
        };

        // maximumWeight 設定 cache 的 weight 總和限制
        // weigher 指定 weight function class
        LoadingCache<String, String> cache;
        cache = CacheBuilder.newBuilder()
                .maximumWeight(15)
                .weigher(weighByLength)
                .build(loader);

        cache.getUnchecked("first");
        cache.getUnchecked("second");
        cache.getUnchecked("third");
        // 要加入 third 時,會檢查 weight 總和,這邊總和會變成 16 就超過上限 15,因此 first 會被刪除
        assertEquals(2, cache.size());
        cache.getUnchecked("last");
        assertEquals(3, cache.size());
        assertNull(cache.getIfPresent("first"));
        assertEquals("LAST", cache.getIfPresent("last"));
    }

    @Test
    public void time_thenEviction()
            throws InterruptedException {
        CacheLoader<String, String> loader = createLoader();

        // 當 cached object 已經 idle 2ms 後,就被移除
        // expireAfterAccess
        LoadingCache<String, String> cache;
        cache = CacheBuilder.newBuilder()
                .expireAfterAccess(2, TimeUnit.MILLISECONDS)
                .build(loader);

        cache.getUnchecked("hello");
        assertEquals(1, cache.size());

        cache.getUnchecked("hello");
        Thread.sleep(300);

        cache.getUnchecked("test");
        assertEquals(1, cache.size());
        assertNull(cache.getIfPresent("hello"));

        /////
        // 當 cached object 儲存 2ms 後,就被移除
        // expireAfterWrite
        LoadingCache<String, String> cache2;
        cache2 = CacheBuilder.newBuilder()
                .expireAfterWrite(2,TimeUnit.MILLISECONDS)
                .build(loader);

        cache2.getUnchecked("hello");
        assertEquals(1, cache2.size());
        Thread.sleep(300);
        cache2.getUnchecked("test");
        assertEquals(1, cache2.size());
        assertNull(cache2.getIfPresent("hello"));
    }

    CacheLoader<String, String> createLoader() {
        CacheLoader<String, String> loader;
        loader = new CacheLoader<String, String>() {
            @Override
            public String load(String key) {
                return key.toUpperCase();
            }
        };
        return loader;
    }

Weak Keys

    @Test
    public void whenWeakKeyHasNoRef_thenRemoveFromCache() {
        CacheLoader<String, String> loader;
        loader = new CacheLoader<String, String>() {
            @Override
            public String load(String key) {
                return key.toUpperCase();
            }
        };

        // weakKeys 定義 cache 的 key 是用 weakReference
        // garbage collector 可在 key 不被 referenced 的時候,將 key 回收
        LoadingCache<String, String> cache;
        cache = CacheBuilder.newBuilder().weakKeys().build(loader);
    }

Soft Values

    @Test
    public void whenSoftValue_thenRemoveFromCache() {
        CacheLoader<String, String> loader;
        loader = new CacheLoader<String, String>() {
            @Override
            public String load(String key) {
                return key.toUpperCase();
            }
        };

        // softValues
        // 可讓 garbage collector 回收 cached values
        // 但如果使用太多 soft references 會影響效能,故還是建議使用 maxmimumSize
        LoadingCache<String, String> cache;
        cache = CacheBuilder.newBuilder().softValues().build(loader);
    }

null Values

    @Test
    public void whenNullValue_thenOptional() {
        // 如果要 laod null value,預設會 throw exception
        // 但如果一定要使用 null value,可利用 Optional
        CacheLoader<String, Optional<String>> loader;
        loader = new CacheLoader<String, Optional<String>>() {
            @Override
            public Optional<String> load(String key) {
                return Optional.fromNullable(getSuffix(key));
            }
        };

        LoadingCache<String, Optional<String>> cache;
        cache = CacheBuilder.newBuilder().build(loader);

        assertEquals("txt", cache.getUnchecked("text.txt").get());
        assertFalse(cache.getUnchecked("hello").isPresent());
    }
    private String getSuffix(final String str) {
        int lastIndex = str.lastIndexOf('.');
        if (lastIndex == -1) {
            return null;
        }
        return str.substring(lastIndex + 1);
    }

refresh Cache

更新 cache value 的方法,分為 manual/auto 兩種

    @Test
    public void whenLiveTimeEnd_thenRefresh() {
        CacheLoader<String, String> loader;
        loader = new CacheLoader<String, String>() {
            @Override
            public String load(String key) {
                return key.toUpperCase();
            }
        };

        LoadingCache<String, String> cache0;
        cache0 = CacheBuilder.newBuilder().build(loader);
        cache0.getUnchecked("hello");
        try {
            // get 會取得舊的 cachec value
            String value = cache0.get("hello");
        } catch (ExecutionException e) {
            throw new RuntimeException(e);
        }
        // 呼叫 refresh,(非同步) 更新 cached value
        cache0.refresh("hello");

        //////////////
        // CacheBuilder.refreshAfterWrite(duration) 可自動 refresh cached value
        // 如果 1min 後沒有 get 這個 key,當 cached value 有舊的值,其他 threads 會回傳舊值
        // 如果 cached value 沒有值,就更新 value,其他 threads 會等待
        LoadingCache<String, String> cache;
        cache = CacheBuilder.newBuilder()
                .refreshAfterWrite( 1,TimeUnit.MINUTES)
                .build(loader);
    }

Preload the Cache

    @Test
    public void whenPreloadCache_thenUsePutAll() {
        CacheLoader<String, String> loader;
        loader = new CacheLoader<String, String>() {
            @Override
            public String load(String key) {
                return key.toUpperCase();
            }
        };

        LoadingCache<String, String> cache;
        cache = CacheBuilder.newBuilder().build(loader);

        // 透過 Map 的 putAll,一次加入多個 cache values
        Map<String, String> map = new HashMap<String, String>();
        map.put("first", "FIRST");
        map.put("second", "SECOND");
        cache.putAll(map);

        assertEquals(2, cache.size());
    }

RemovalNotification

    @Test
    public void whenEntryRemovedFromCache_thenNotify() {
        CacheLoader<String, String> loader;
        loader = new CacheLoader<String, String>() {
            @Override
            public String load(final String key) {
                return key.toUpperCase();
            }
        };

        // RemovalListener 可在 cachec value 被移除時,收到通知
        RemovalListener<String, String> listener;
        listener = new RemovalListener<String, String>() {
            @Override
            public void onRemoval(RemovalNotification<String, String> n){
                if (n.wasEvicted()) {
                    String cause = n.getCause().name();
                    assertEquals(RemovalCause.SIZE.toString(),cause);
                }
            }
        };

        LoadingCache<String, String> cache;
        cache = CacheBuilder.newBuilder()
                .maximumSize(3)
                .removalListener(listener)
                .build(loader);

        cache.getUnchecked("first");
        cache.getUnchecked("second");
        cache.getUnchecked("third");
        cache.getUnchecked("last");
        assertEquals(3, cache.size());
    }

Others

  • Cache 是 thread-safe

  • 可透過 put(key,value) ,不透過 CacheLoader 直接寫入 cache

    cache.put("key", "value");
    assertEquals("value", cache.getUnchecked("key"));
  • 可用 CacheStats ( hitRate()missRate(), ..) 量測 performance

References

Guava Cache | Baeldung

CachesExplained · google/guava Wiki · GitHub

2024/04/15

Guava Collection Utilities: Maps, Multisets, Multimaps, Tables

Maps

    @Test
    public void maps() {
        Map<String, Integer> left = ImmutableMap.of("a", 1, "b", 2, "c", 3);
        Map<String, Integer> right = ImmutableMap.of("b", 2, "c", 4, "d", 5);
        MapDifference<String, Integer> diff = Maps.difference(left, right);

        // 兩個 map 都有出現的 key-value pair
        Map<String, Integer> entriesInCommon = diff.entriesInCommon(); // {"b" => 2}
        assertEquals("{b=2}", entriesInCommon.toString());

        // 有相同的 key,不同的 values
        Map<String, MapDifference.ValueDifference<Integer>> entriesDiffering = diff.entriesDiffering(); // {"c" => (3, 4)}
        assertEquals("{c=(3, 4)}", entriesDiffering.toString());

        // 出現在 left,沒有出現在 right 的 key-value pair
        Map<String, Integer> entriesOnlyOnLeft = diff.entriesOnlyOnLeft(); // {"a" => 1}
        assertEquals("{a=1}", entriesOnlyOnLeft.toString());
        // 出現在 right,沒有出現在 left
        Map<String, Integer> entriesOnlyOnRight = diff.entriesOnlyOnRight(); // {"d" => 5}
        assertEquals("{d=5}", entriesOnlyOnRight.toString());
    }

uniqueIndex

    @Test
    public void maps_uniqueIndex() {
        // nickname屬性能唯一確定一個WebUser
        ArrayList<User> users = Lists.newArrayList(new User(1,"one"),new User(2,"two"),new User(3,"three"),new User(4,"four"));
        // 以 name 為 key,User為值的map
        ImmutableMap<String, User> map = Maps.uniqueIndex(users,new com.google.common.base.Function<User, String>() {
            @Override
            public String apply(User user) {
                return user.getName();
            }
        });
        System.out.println("map:" + map);
        System.out.println("name:" + map.get("two").getName());

        assertEquals("{one=User(id=1,name=one), two=User(id=2,name=two), three=User(id=3,name=three), four=User(id=4,name=four)}", map.toString());
    }
    class User {
        private int id;
        private String name;

        public User(int id, String name) {
            this.id = id;
            this.name = name;
        }

        public String getName() {
            return this.name;
        }

        public String toString() {
            return "User(id="+id+",name="+name+")";
        }
    }

Multisets

    @Test
    public void multisets() {
        Multiset<String> multiset1 = HashMultiset.create();
        multiset1.add("a", 2);

        Multiset<String> multiset2 = HashMultiset.create();
        multiset2.add("a", 5);

        // returns true: all unique elements are contained,
        boolean containsAll = multiset1.containsAll(multiset2);
        assertTrue(containsAll);

        // containsOccurrences(Multiset sup, Multiset sub)
        // if sub.count(o) <= sup.count(o) for all o  -> return true
        // even though multiset1.count("a") == 2 < multiset2.count("a") == 5
        // supCountA=2, subCountA=5
        // System.out.println("multiset1 CountA="+multiset1.count("a")+", multiset2 CountA="+multiset2.count("a"));
        boolean containsOccurrences = Multisets.containsOccurrences(multiset1, multiset2); // returns false
        assertFalse(containsOccurrences);

        // retainOccurrences(Multiset removeFrom, Multiset toRetain)
        // 確保 removeFrom.count(o) <= toRetain.count(o) for all o
        boolean retainOccurrences = Multisets.retainOccurrences(multiset1, multiset2);
        assertFalse(retainOccurrences);
        System.out.println("multiset1 CountA="+multiset1.count("a")+", multiset2 CountA="+multiset2.count("a"));

        // intersection of two multisets
        Multiset<String> intersection = Multisets.intersection(multiset1, multiset2);
        assertEquals(2, intersection.count("a"));

        // removeOccurrences(Multiset removeFrom, Multiset toRemove)
        // 自 removeFrom 中,移除 toRemove 的 element 出現的次數
        Multisets.removeOccurrences(multiset2, multiset1); // multiset2 now contains 3 occurrences of "a"
        assertEquals(3, multiset2.count("a"));

        // 移除 multiset2 中所有的元素
        multiset2.removeAll(multiset1); // removes all occurrences of "a" from multiset2, even though multiset1.count("a") == 2
        assertTrue(multiset2.isEmpty()); // returns true


        Multiset<String> multiset = HashMultiset.create();
        multiset.add("a", 3);
        multiset.add("b", 5);
        multiset.add("c", 1);

        // 次數從大到小的順序,排列 multiset
        ImmutableMultiset<String> highestCountFirst = Multisets.copyHighestCountFirst(multiset);
        assertEquals("[b x 5, a x 3, c]", highestCountFirst.toString());
    }

Multimaps

    @Test
    public void mutlimaps() {
        ImmutableSet<String> digits = ImmutableSet.of(
                "zero", "one", "two", "three", "four",
                "five", "six", "seven", "eight", "nine");
        Function<String, Integer> lengthFunction = new Function<String, Integer>() {
            public Integer apply(String string) {
                return string.length();
            }
        };
        //  Multimaps.index(Iterable, Function)
        // 以 Function apply 的結果分類,將相同字串長度的元素,放在一起
        ImmutableListMultimap<Integer, String> digitsByLength = Multimaps.index(digits, lengthFunction);
        /*
         * digitsByLength maps:
         *  3 => {"one", "two", "six"}
         *  4 => {"zero", "four", "five", "nine"}
         *  5 => {"three", "seven", "eight"}
         */
        assertEquals("{4=[zero, four, five, nine], 3=[one, two, six], 5=[three, seven, eight]}", digitsByLength.toString());

        ////////
        //  Multimaps.invertFrom(Multimap toInvert, Multimap dest)
        //  如果要得到 Immutablemiltimap 就呼叫 ImmutableMultimap.inverse()
        ArrayListMultimap<String, Integer> multimap = ArrayListMultimap.create();
        multimap.putAll("b", Ints.asList(2, 4, 6));
        multimap.putAll("a", Ints.asList(4, 2, 1));
        multimap.putAll("c", Ints.asList(2, 5, 3));

        TreeMultimap<Integer, String> inverse = Multimaps.invertFrom(multimap, TreeMultimap.<Integer, String>create());
        // note that we choose the implementation, so if we use a TreeMultimap, we get results in order
        /*
         * inverse maps:
         *  1 => {"a"}
         *  2 => {"a", "b", "c"}
         *  3 => {"c"}
         *  4 => {"a", "b"}
         *  5 => {"c"}
         *  6 => {"b"}
         */
        assertEquals("{1=[a], 2=[a, b, c], 3=[c], 4=[a, b], 5=[c], 6=[b]}", inverse.toString());

        /////
        // forMap(Map)  將 Map 轉換為 SetMultimap
        // 特別適合用在 Multimaps.invertFrom
        Map<String, Integer> map = ImmutableMap.of("a", 1, "b", 1, "c", 2);
        SetMultimap<String, Integer> multimap2 = Multimaps.forMap(map);
        // multimap maps ["a" => {1}, "b" => {1}, "c" => {2}]
        Multimap<Integer, String> inverse2 = Multimaps.invertFrom(multimap2, HashMultimap.<Integer, String> create());
        // inverse maps [1 => {"a", "b"}, 2 => {"c"}]
        assertEquals("{1=[a, b], 2=[c]}", inverse2.toString());
    }

Tables

    @Test
    public void tables() {
        // Tables.newCustomTable(Map, Supplier<Map>)
        Table<String, Character, Integer> table = Tables.newCustomTable(
                Maps.<String, Map<Character, Integer>>newLinkedHashMap(),
                new Supplier<Map<Character, Integer>>() {
                    public Map<Character, Integer> get() {
                        return Maps.newLinkedHashMap();
                    }
                });
        table.put("a", 'x', 1);
        table.put("a", 'y', 2);
        table.put("a", 'z', 3);
        table.put("b", 'x', 4);
        table.put("b", 'y', 5);
        table.put("b", 'z', 6);
        assertEquals("{a={x=1, y=2, z=3}, b={x=4, y=5, z=6}}", table.toString());

        // transpose 轉置矩陣
        assertEquals("{x={a=1, b=4}, y={a=2, b=5}, z={a=3, b=6}}", Tables.transpose(table).toString());
    }

References

CollectionUtilitiesExplained · google/guava Wiki · GitHub

2024/04/08

Guava Collection Utilities: Lists, Sets

Guava 在 Collection interfaces 有對應的 utility class,這些 utilities 是 static method,可直接使用。

Interface JDK or Guava? Corresponding Guava utility class
Collection JDK Collections2
List JDK Lists
Set JDK Sets
SortedSet JDK Sets
Map JDK Maps
SortedMap JDK Maps
Queue JDK Queues
Multiset Guava Multisets
Multimap Guava Multimaps
BiMap Guava Maps
Table Guava Tables

Static Contructor

    @Test
    public void static_constructor() {
        //JDK 的 list 必須建立後再加入元素
        List<String> list = new ArrayList<>();
        list.add("item1");
        list.add("item2");

        // Guava 在建立物件時,可直接填寫 init data
        Set<String> sets = Sets.newHashSet("alpha", "beta", "gamma");
        List<String> list2 = Lists.newArrayList("alpha", "beta", "gamma");

        // Guava 在 init 時,可直接設定 size
        List<String> exactly100 = Lists.newArrayListWithCapacity(100);
        List<String> approx100 = Lists.newArrayListWithExpectedSize(100);
        Set<String> approx100Set = Sets.newHashSetWithExpectedSize(100);

        // Guava 新的 Collection 類別,不提供 constructor,而是提供一個 factory method: create
        Multiset<String> multiset = HashMultiset.create();
    }

Iteratables

Iterables 封裝了 Iterable,提供 fluent 語法

    @Test
    public void iterables() {
        //// concat
        //// concat 可將兩個 collections 連接在一起
        Iterable<Integer> concatenated = Iterables.concat(
                Ints.asList(1, 2, 3),
                Ints.asList(5, 5, 6));
        // concatenated has elements 1, 2, 3, 4, 5, 6
        assertEquals("[1, 2, 3, 5, 5, 6]", concatenated.toString());

        //// concat 在不同類別的 list,不能連接在一起
//        Iterable<Integer> concatenated2 = Iterables.concat(
//                Ints.asList(1, 2, 3),
//                Lists.newArrayList("alpha", "beta", "gamma"));

        //// getFirst: first element, 第二個參數是 預設值
        //// getLast: last element, 失敗時會 throw NoSuchElementException
        Integer firstElement = Iterables.getFirst(concatenated, null);
        Integer lastAdded = Iterables.getLast(concatenated);
        assertEquals(firstElement.intValue(), 1);
        assertEquals(lastAdded.intValue(), 6);

        /// getOnlyElement: the only element in Iterable
        Iterable<Integer> iterable2 = () -> Arrays.asList(100).iterator();
        Iterable<Integer> iterable3 = Collections.singleton(200);
        Exception exception = assertThrows(IllegalArgumentException.class, () -> {
            Integer theElement = Iterables.getOnlyElement(concatenated);
        });
        assertEquals(Iterables.getOnlyElement(iterable2).intValue(), 100);

        //// frequency:  查詢某個 element 出現的次數
        assertEquals(2, Iterables.frequency(concatenated, 5));

        //// partition:  分割成多個 Iterable
        Iterable<List<Integer>> list = Iterables.partition(concatenated, 2);
//        System.out.println("list="+ list);
        assertEquals("[[1, 2], [3, 5], [5, 6]]", list.toString());
        assertEquals("[1, 2]", Iterables.getFirst(list, null).toString());
    }

Iterables 提供跟 Collection 類似的一些 methods

    @Test
    public void iterables_collection_like() {
        // Iterables 提供這些類似 Collection 的 methods
        // 1. addAll
        // 2. contains
        // 3. removeAll
        // 4. retainAll
        // 5. size
        // 6. toArray
        // 7. isEmpty
        // 8. get
        // 9. toString
        ArrayList<Integer> list1 = Lists.newArrayList(1, 2);
        ArrayList<Integer> list2 = Lists.newArrayList(100, 101);
        boolean changed = Iterables.addAll( list1 , list2);
        assertTrue( changed );
        assertEquals("[1, 2, 100, 101]", list1.toString());

        assertTrue( list1.contains(2) );
        assertEquals(list1.size(), 4);
        assertFalse( list1.isEmpty() );

        assertEquals(1, list1.get(0).intValue());
    }

FluentIterable 有幾個 method,可產生 immutable collection

Result Type Method
ImmutableList toList()
ImmutableList toSortedList()
ImmutableSet toSet()
ImmutableSortedSet toSortedSet()
ImmutableMultiset toMultiset()
ImmutableMap toMap()
    @Test
    public void FluentIterable_immutable() {
        // FluentIterable
        ImmutableList<String> list1 =
                FluentIterable.from( Arrays.asList(1,2,3) )
                .transform( Functions.toStringFunction() )
                .limit(10)
                .toList();
        assertEquals( 3, list1.size());
    }

Lists

    @Test
    public void lists() {
        List<Integer> countUp = Ints.asList(1, 2, 3);

        // Lists.reverse: reverse a list
        List<Integer> countDown = Lists.reverse(countUp); // {3, 2, 1}
        // 將 list 以 size 分割成多個 sublist
        List<List<Integer>> parts = Lists.partition(countUp, 2); // {{1, 2}, {3}}

        assertEquals("[1, 2, 3]", countUp.toString());
        assertEquals("[3, 2, 1]", countDown.toString());
        assertEquals("[[1, 2], [3]]", parts.toString());

        // Lists 可產生 ArrayList 及 LinkedList
        // Lists.newArrayList()
        // Lists.newLinkedList()
        List<Integer> list1 = Lists.newArrayList(1,3,2,4);
        List<Integer> list2 = Lists.newLinkedList(Ints.asList(1, 3, 2));
        assertEquals("[1, 3, 2, 4]", list1.toString());
        assertEquals("[1, 3, 2]", list2.toString());

        // 透過 Collections.max 找到最大的元素, Collections.min 最小的元素
        int maxElement = Collections.max(Arrays.asList(1,3,2,4));
        int minElement = Collections.min(Arrays.asList(1,3,2,4));
        assertEquals(4, maxElement);
        assertEquals(1, minElement);
    }

Sets

    @Test
    public void sets() {
        Set<String> set1 = ImmutableSet.of("a", "b", "c");
        Set<String> set2 = ImmutableSet.of("b", "c", "d");

        // 聯集 union of two sets
        Set<String> union1 = Sets.union(set1, set2);
        // 回傳 SetView,可再轉換為 immutable set 使用
        Sets.SetView<String> union2 = Sets.union(set1, set2);
        ImmutableSet<String> immutableSet = union2.immutableCopy();
        assertEquals("[a, b, c, d]", union1.toString());
        assertEquals("[a, b, c, d]", immutableSet.toString());

        // 交集 intersection of two sets
        Sets.SetView<String> intersection = Sets.intersection(set1, set2);
        // I can use intersection as a Set directly, but copying it can be more efficient if I use it a lot.
        assertEquals("[b, c]", intersection.toString());

        // 差集 difference
        Set<String> diff = Sets.difference(set1, set2);
        assertEquals("[a]", diff.toString());

        // symmetricDifference
        Set<String> diff2 = Sets.symmetricDifference(set1, set2);
        assertEquals("[a, d]", diff2.toString());

        // cartesianProduct: Cartesian Product 笛卡兒積
        Set<List<String>> product = Sets.cartesianProduct(set1, set2);
        assertEquals("[[a, b], [a, c], [a, d], [b, b], [b, c], [b, d], [c, b], [c, c], [c, d]]", product.toString());

        // powerSet: 找到所有可能的 subsets
        Set<Set<String>> powerset = Sets.powerSet(set1);
        assertTrue(powerset.contains(ImmutableSet.of("a")));
        assertTrue(powerset.contains(ImmutableSet.of("b")));
        assertTrue(powerset.contains(ImmutableSet.of("a", "b")));
        assertTrue(powerset.contains(ImmutableSet.of("a", "b", "c")));
        assertEquals(8, powerset.size());
    }

References

CollectionUtilitiesExplained · google/guava Wiki · GitHub