Natural Pleistocene Monsters
    Share your code. npm Orgs help your team discover, share, and reuse code. Create a free org »

    microcrawler-workerpublic

    Microcrawler Worker

    Status

    Build Status License Dependency Status

    Prerequisities

    Getting started

    Clone Sources

    git clone https://github.com/AplloCrawler/microcrawler-worker.git
    

    Enter Directory with Sources

    cd microcrawler-worker
    

    Install Dependencies

    npm install
    

    Run Microcrawler Worker

    bin/microcrawler-worker.js
    

    Usage

    Show Help

    $ ./bin/microcrawler-worker.js -h
    
      Usage: microcrawler-worker [options]
    
      Options:
    
        -h, --help                           output usage information
        -V, --version                        output the version number
        -c, --channel <CHANNEL>              Channel to connect to, default: worker:lobby
        --heartbeat-interval <MILLISECONDS>  Heartbeat interval in milliseconds, default: 10000
        -i, --interactive                    Run interactive mode
        -u, --url <URL>                      URL to connect to, default: ws://localhost:4000/socket
        -t, --token <TOKEN>                  Token used for authorization, default: null
    

    Print Version

    $ ./bin/microcrawler-worker.js --version
    0.0.1
    

    Connect to Default URL

    $ ./bin/microcrawler-worker.js
    Connecting to "ws://localhost:4000/socket"
    

    Connect to Custom URL

    $ ./bin/microcrawler-worker.js -u ws://example.com/socket
    Connecting to "ws://example.com/socket"
    

    Sample Output

    $ ./bin/microcrawler-worker.js --heartbeat-interval 10000
    Connecting to "ws://localhost:4000/socket"
    Received ok
    {
        "msg": "Welcome!"
    }
    Received event - pong
    {
        "version": "0.0.1",
        "os": {
            "uptime": 139871,
            "platform": "darwin",
            "mem": {
                "total": 8589934592,
                "free": 134340608
            },
            "load": [
                2.14794921875,
                2.11328125,
                2.318359375
            ],
            "hostname": "kx-mac.local",
            "endian": "LE",
            "cpus": [
                {
                    "times": {
                        "user": 17079150,
                        "sys": 8949760,
                        "nice": 0,
                        "irq": 0,
                        "idle": 65412150
                    },
                    "speed": 1700,
                    "model": "Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz"
                },
                {
                    "times": {
                        "user": 7933380,
                        "sys": 3558830,
                        "nice": 0,
                        "irq": 0,
                        "idle": 79947730
                    },
                    "speed": 1700,
                    "model": "Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz"
                },
                {
                    "times": {
                        "user": 17069960,
                        "sys": 7510340,
                        "nice": 0,
                        "irq": 0,
                        "idle": 66859640
                    },
                    "speed": 1700,
                    "model": "Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz"
                },
                {
                    "times": {
                        "user": 8092410,
                        "sys": 3646170,
                        "nice": 0,
                        "irq": 0,
                        "idle": 79701350
                    },
                    "speed": 1700,
                    "model": "Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz"
                }
            ]
        },
        "name": "microcrawler-worker",
        "msg": "I am still alive!",
        "id": 0,
        "ts": 1475360236777
    }
    

    Interactive Mode

    Server

    $ iex -S mix phoenix.server
    Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
    
    [info] Running MicrocrawlerWebapp.Endpoint with Cowboy using http://localhost:4000
    Interactive Elixir (1.3.3) - press Ctrl+C to exit (type h() ENTER for help)
    iex(1)> 05 Oct 20:54:35 - info: compiled 6 files into 2 files, copied 6 in 1.6 sec
    [info] JOIN worker:lobby to MicrocrawlerWebapp.WorkerChannel
      Transport:  Phoenix.Transports.WebSocket
      Parameters: %{"token" => nil}
    Received join - worker:lobby
    {
      "token": null
    }
    #PID<0.366.0>
    [info] Replied worker:lobby :ok
    {:basic_consume_ok, %{consumer_tag: "amq.ctag-Rb-fiGVc4xTncGoOKjxyKA"}}
    #PID<0.366.0>
    Received event - msg
    {
      "msg": "Hello World!"
    }
    Received event - msg
    "Hi, how are you?"
    Received event - msg
    "Received my message?"
    Received event - msg
    42
    Received event - msg
    {
      "abc": "123"
    }
    Received event - msg
    {
      "pole": [
        1,
        2,
        3,
        4
      ],
      "msg": "to pole je fakt urodny"
    }
    {:shutdown, :closed}
    #PID<0.366.0>
    %Phoenix.Socket{assigns: %{rabb_chan: %AMQP.Channel{conn: %AMQP.Connection{pid: #PID<0.369.0>},
        pid: #PID<0.379.0>}, rabb_conn: %AMQP.Connection{pid: #PID<0.369.0>}},
     channel: MicrocrawlerWebapp.WorkerChannel, channel_pid: #PID<0.366.0>,
     endpoint: MicrocrawlerWebapp.Endpoint, handler: MicrocrawlerWebapp.UserSocket,
     id: nil, joined: true, pubsub_server: MicrocrawlerWebapp.PubSub, ref: nil,
     serializer: Phoenix.Transports.WebSocketSerializer, topic: "worker:lobby",
     transport: Phoenix.Transports.WebSocket, transport_name: :websocket,
     transport_pid: #PID<0.364.0>}
    

    Client

    $ ./bin/microcrawler-worker.js -i --heartbeat-interval 3600000
    Connecting to "ws://localhost:4000/socket"
    Running in interactive mode.
    Type "quit" or press ctrl+c twice to exit.
    msg> Received ok
    {
        "msg": "Welcome!"
    }
    msg> Hi, how are you?
    msg> Received my message?
    msg> 42
    msg> {"abc": "123"}
    msg> {"pole": [1, 2, 3, 4], "msg": "to pole je fakt urodny"}
    msg> quit
    $
    

    install

    npm i microcrawler-worker

    Downloadsweekly downloads

    23

    version

    0.0.18

    license

    MIT

    repository

    githubgithub

    last publish

    collaborators

    • avatar