2019/06/03

Interoperability and Meta Programming

julia 可跟 C, Python 的程式互動。另外 juila 支援 meta programming 的形式。

Interacting with OS

在 REPL 可用 ; 進入 shell，執行 shell commands，如果在 julia，則是直接用 pwd()

shell> pwd
/Users/user

julia> pwd()
"/Users/user"
filesystem operations

存取檔案的 functions

  • homedir(), joinpath()

    為支援 linux, windows，使用 joinpath() 產生檔案的路徑

julia> homedir()
"/Users/user"

julia> joinpath(homedir(), "Documents")
"/Users/user/Documents"

  • pwd(), cd()

    取得目前的目錄，以及切換目錄

    cd("../")

  • readdir()

    取得目錄內的檔案列表，包含目錄及檔案

    readdir(".")

  • mkdir()

    d = joinpath(homedir(), "LearningJulia")
mkdir(d)

  • stat

    stat("somefile") 會取得 StatStruct(mode=0o100644, size=28676)，而 StatStruct 裡面有這些欄位

    ref: https://github.com/JuliaLang/julia/blob/master/base/stat.jl

    ex: stat("somefile").size

    Name Description
    size The size (in bytes) of the file
    device ID of the device that contains the file
    inode The inode number of the file
    mode The protection mode of the file
    nlink The number of hard links to the file
    uid The user id of the owner of the file
    gid The group id of the file owner
    rdev If this file refers to a device, the ID of the device it refers to
    blksize The file-system preferred block size for the file
    blocks The number of such blocks allocated
    mtime Unix timestamp of when the file was last modified
    ctime Unix timestamp of when the file was created
    • cp, mv
    julia> cp("test", "test1")
"test1"

julia> isfile("test1")
true

julia> mv("test1", "test2")
"test2"

  • isdir, homedir, basename, dirname, and splitdir

    julia> homedir()
"/Users/user"

julia> joinpath(homedir(), "Documents")
"/Users/user/Documents"

julia> isdir(joinpath(homedir(), "Documents"))
true

julia> basename(homedir())
"user"

julia> dirname(homedir())
"/Users"

julia> splitdir(homedir())
("/Users", "user")

  • mkpath, ispath, abspath, and joinpath

    julia> ispath(homedir())
true

julia> abspath(".")
"/Users/user/Downloads/"

julia> joinpath(homedir(), "Documents")
"/Users/user/Documents"

julia> mkpath(joinpath("Users","adm"))
"Users/adm"

julia> for (root, dirs, files) in walkdir("Users")
               println("Directories in $root")
       end
Directories in Users
Directories in Users/adm
I/O Operations

STDOUT, STDERR, STDIN 是三個 global variables，分別為 standard output, error, input stram

open(), close(), write, read

julia> file = open("sample.txt")
IOStream(<file sample.txt>)

julia> file
IOStream(<file sample.txt>)

julia> lines = readlines(file)
2-element Array{String,1}:
 "Hi there!"
 "Learning Julia is so much fun."

julia> write("sample.txt", "hi how are you doing?")
21

julia> read("sample.txt")
21-element Array{UInt8,1}:

julia> readline("sample.txt")
"hi how are you doing?"

julia> close(file)

讀取某個文字檔，計算裡面有幾個 "Julia" 這個 word

# Arguments
# ARGS[0] is the command itself
in_file = ARGS[1]
out_file = ARGS[2]

# Keeping track using a counter
counter = 0
for line in eachline(in_file)
    for word in split(line)
        if word == "Julia"
            global counter += 1
        end
    end
end

# Write the contents to the o/p file
write(out_file, "The count for the word julia is $counter")

# Finally, read the contents to inform the user
for line in readlines(out_file)
    println(line)
end
shell> julia sample.jl sample.txt out.txt
The count for the word julia is 4

Calling C and Python

Calling C from Julia

compile (ex: LLVM) 需要知道什麼，才能由 Julia 呼叫 C

  • Name of the library
  • Name of the function
  • Number and types of the arguments (也稱為 Arity)
  • return type of the function
  • values of the arguments passed

julia 使用 ccall((:name,"lib"), return_type, (arg1_type, arg2_type...), arg1, arg2) 處理

呼叫 C 標準函式庫的 clock function

julia> ccall((:clock, :libc), Int64, ())
3206897

julia> ccall((:clock, :libc), Cint, ())
3213482

julia> Int32 == Cint
true

Cint 是 C 的資料型別，等於 signed int c-type，另外還有 Cint, Cuint, Clong, Culong, and Cchar

呼叫 getenv，取得 SHELL value，Ptr{Cchar} 是參數的資料型別，後面 return value 資料型別也是 Ptr{Cchar}

julia> syspath = ccall( (:getenv, :libc), Ptr{Cchar}, (Ptr{Cchar},), "SHELL")
Ptr{Int8} @0x00007ffee37b857a

julia> unsafe_string(syspath)
"/bin/bash"

C 的 struct 可以替換為 julia 的 composite types

Calling Python form Julia

使用 PyCall

julia> using Pkg

julia> Pkg.add("PyCall")

julia> using PyCall

julia> py"len('julia')"
5

julia> py"str(5)"
"5"

julia> py"""
       print('hello world')
       """
hello world

如果要使用 python 的 built-in data type (ex: dict)，可使用 pybuiltin

julia> pybuiltin(:dict)(a=1,b=2)
Dict{Any,Any} with 2 entries:
  "b" => 2
  "a" => 1

julia> d = pycall(pybuiltin("dict"), Any, a=1, b=2)
PyObject {'b': 2, 'a': 1}

julia> typeof(d)
PyObject

# 利用 PyDict 轉換為 julia dict object
julia> julia_dictionary = PyDict{Symbol, Int64}(pycall(pybuiltin("dict"), Any, a=1, b=2))
PyDict{Symbol,Int64,true} with 2 entries:
  :b => 2
  :a => 1

julia> typeof(julia_dictionary)
PyDict{Symbol,Int64,true}

julia> julia_dictionary[:a]
1

# 產生一個 kw 參數的 function
julia> f(; a=0, b=0) = [10a, b]
f (generic function with 1 method)

# 以 dict 呼叫 function
julia> f(;julia_dictionary...)
2-element Array{Int64,1}:
 10
  2

Expressions

這是 metaprogramming 的功能，這是參考 LISP 的功能

首先說明 julia 如何 interprets a code

julia> code = "println(\"hello world \")"
"println(\"hello world \")"

julia> expression = Meta.parse(code)
:(println("hello world "))

julia> typeof(expression)
Expr
julia> expression.args
2-element Array{Any,1}:
 :println
 "hello world "

julia> expression.head
:call

julia> dump(expression)
Expr
  head: Symbol call
  args: Array{Any}((2,))
    1: Symbol println
    2: String "hello world "

產生 expression，用 eval 運算， :call 是 symbol，表示 head of the expression

julia> sample_expr = Expr(:call, +, 10, 20)
:((+)(10, 20))

julia> eval(sample_expr)
30

julia> sample_expr.args
3-element Array{Any,1}:
   +
 10
 20

將參數替換為 :x, :y 變數，也可以直接寫 x, y

julia> x = 10; y =10
10

julia> sample_expr = Expr(:call, +, :x, :y)
:((+)(x, y))

julia> sample_expr = Expr(:call, +, x, y)
:((+)(10, 10))

julia> eval(sample_expr)
20

也可以使用 $

julia> x = 10; y=10
10

julia> e = :($x + $y)
:(10 + 10)

julia> eval(e)
20

所以有 : 跟 $ 兩種方式可產生 Expr object

  • Using quotes(:) at runtime
  • Using dollar($) ar parse time

另外還有一個方式，是使用 quote 這個 keyword。quote 跟 : 的差別，是用 quote 可讓程式碼排列更好，比較容易實作

julia> quote
          30 * 100
       end
quote
    #= REPL[88]:2 =#
    30 * 100
end

julia> eval(ans)
3000

julia> :(30 * 100)
:(30 * 100)

julia> eval(ans)
3000

Macros

類似 function，但 function 使用 variables 為參數，而 macros 使用 expressions，並回傳 modified expressions，呼叫 macro 是用 @

macro NAME
    # some custom code
    # return modified expression
end

因為 REPL 建立的 macro 是在 Main module 裡面

julia> macro HELLO(name)
         :( println("Hello! ", $name))
       end
@HELLO (macro with 1 method)

julia> @HELLO("world")
Hello! world

julia> macroexpand(Main, :(@HELLO("world")))
:((Main.println)("Hello! ", "world"))
why metaprogramming?

metaprogramming 可省略很多重複的程式碼

for sym in [:foo, :bar, :baz]
    @eval function $(Symbol(sym))(n::Int64)
        for i in 1:n
            println( $(string(sym)) )
        end
    end
end

可這樣使用

foo(1)
bar(2)
baz(3)
for header in [:h1, :h2, :h3, :h4, :h5, :h6]
     @eval function $(Symbol(header))(text::String)
         println("<" * $(string(header))* ">"  * " $text " * "</" * $(string(header))* ">")
     end
 end

h1("Hello world!")
# <h1> Hello world! </h1>

h3("Hello world!")
# <h3> Hello world! </h3>
Built-in macros

以下是所有內建的 macros

julia> @
@MIME_str      @boundscheck    @edit           @html_str       @nospecialize   @text_str
@__DIR__       @cfunction      @elapsed        @inbounds       @polly          @threadcall
@__FILE__      @cmd            @enum           @info           @r_str          @time
@__LINE__      @code_llvm      @error          @inline         @raw_str        @timed
@__MODULE__    @code_lowered   @eval           @int128_str     @s_str          @timev
@__dot__       @code_native    @evalpoly       @isdefined      @show           @uint128_str
@allocated     @code_typed     @fastmath       @label          @simd           @v_str
@assert        @code_warntype  @functionloc    @less           @specialize     @view
@async         @debug          @generated      @macroexpand    @static         @views
@b_str         @deprecate      @gensym         @macroexpand1   @sync           @warn
@big_str       @doc            @goto           @noinline       @task           @which

  • @time

    可取得執行某一段程式的耗費時間

    julia> function recursive_sum(n)
          if n == 0
              return 0
          else
              return n + recursive_sum(n-1)
          end
       end
recursive_sum (generic function with 1 method)

julia> @time recursive_sum(10000)
  0.005359 seconds (3.22 k allocations: 183.158 KiB)
50005000

  • @elapsed

    類似 @time，但只有回傳時間 in Float64

    julia> @elapsed recursive_sum(10000)
3.0984e-5

julia> @elapsed recursive_sum(10000)
2.4597e-5

  • @show

    會回傳 expression

    julia> @show(println("hello world"))
hello world
println("hello world") = nothing

julia> @show(:(println("hello world")))
$(Expr(:quote, :(println("hello world")))) = :(println("hello world"))
:(println("hello world"))

julia> @show(:(3*2))
$(Expr(:quote, :(3 * 2))) = :(3 * 2)
:(3 * 2)

julia> @show(3*2)
3 * 2 = 6
6

julia> @show(Int64)
Int64 = Int64
Int64

  • @which

    如果有某個 function 有多個 methods，也就是 multiple dispatch 的功能，想知道會呼叫哪一個 method

    julia> function tripple(n::Int64)
          3n
       end
tripple (generic function with 1 method)

julia> function tripple(n::Float64)
          3n
       end
tripple (generic function with 2 methods)

julia> methods(tripple)
# 2 methods for generic function "tripple":
[1] tripple(n::Float64) in Main at REPL[11]:2
[2] tripple(n::Int64) in Main at REPL[10]:2

julia> @which tripple(10)
tripple(n::Int64) in Main at REPL[10]:2

julia> @which tripple(10.0)
tripple(n::Float64) in Main at REPL[11]:2

  • @task

    類似 coroutine，用來產生一個 task，而不是直接執行

    julia> say_hello() = println("hello world")
say_hello (generic function with 1 method)

julia> say_hello_task = @task say_hello()
Task (runnable) @0x0000000113ed9210

julia> istaskstarted(say_hello_task)
false

julia> schedule(say_hello_task)
hello world
Task (queued) @0x0000000113ed9210

julia> yield()

julia> istaskdone(say_hello_task)
true

  • @codellvm, @codelowered, @codetyped,@codenative, and @code_warntype

    瞭解 code 在 julia 中的所有形式

    function fibonacci(n::Int64)
   if n < 2
       n
   else 
       fibonacci(n-1) + fibonacci(n-2)
   end
end

fibonacci(n::Int64) = n < 2 ? n : fibonacci(n-1) + fibonacci(n-2)

# 轉換為 single static assignment，每個變數只會被 assigned 一次，在使用變數前都會先定義
@code_lowered fibonacci(10)

# 
@code_typed fibonacci(10)

@code_warntype fibonacci(10)

# 使用 LLVM C++ API 產生 LLVM intermediate representation
@code_llvm fibonacci(10)

# binary code in memory
@code_native fibonacci(10)

Type introspection

首先定義新的 type: Student，產生兩個物件

struct Student
   name::String
   age::Int64
end

alpha = Student("alpha",24)

beta = Student("beta",25)

可檢查物件的資料型別，或是使用 isa 判斷是否為某個 function 產生的 type

julia> typeof(alpha)
Student

julia> isa(alpha, Student)
true

julia> alpha isa Student
true

reflection

可在 runtime 查詢物件的 attributes。在 julia 產生 function 後，就可以查詢 function 有幾個參數，有哪些 methods。

function calculate_quad(a::Int64,b::Int64,c::Int64,x::Int64)
   return a*x^2 + b*x + c
end

calculate_quad(1,2,3,4)

function calculate_quad(a::Int64,b::Int64,c::Int64,x::Float64)
   return a*x^2 + b*x + c
end

calculate_quad(1,2,3,4.75)

julia> methods(calculate_quad)
# 2 methods for generic function "calculate_quad":
[1] calculate_quad(a::Int64, b::Int64, c::Int64, x::Float64) in Main at REPL[38]:2
[2] calculate_quad(a::Int64, b::Int64, c::Int64, x::Int64) in Main at REPL[36]:2
julia> fieldnames(Student)
(:name, :age)

julia> Student.types
svec(String, Int64)

julia> typeof(Student.types)
Core.SimpleVector

References

Learning Julia

張貼者： 於
標籤：

沒有留言:

張貼留言

訂閱： 張貼留言 (Atom)