TypeScript library to communicate with Scrcpy server.
It's compatible with the official Scrcpy server binaries.
WARNING: The public API is UNSTABLE. Open a GitHub discussion if you have any questions.
- How does Scrcpy work
- What does this package do
- Prepare server binary
- Server versions
- Reading and writing packets
- Always read all streams
- Video/audio stream
- Decode video stream
Scrcpy has two parts: a client and a server.
The server is a Java application (but not an Android app) that runs on the device to capture the video and audio, and send them to the client.
The client receives captured video and audio data, and renders them on computer.
The official Scrcpy client also spawns Google ADB executable to set up the reverse tunnel, push the server binary to the device, and start the server.
This package can't push nor start the server, nor render the video and audio. It only provides APIs to communicate with the server.
@yume-chan/adb-scrcpy
package is a wrapper of this package, provides integration with @yume-chan/adb
package, and can push, start, and stop the server.
This package is compatible with many versions of the official server binary, but it doesn't include the server binary itself.
You can download the server binary from official releases (https://github.com/Genymobile/scrcpy/releases), or use the @yume-chan/fetch-scrcpy-server
package to automate the process.
The server binary is subject to Apache License 2.0.
Scrcpy protocol changes over time, and are usually not backwards compatible. This package supports multiple server versions (ranges), and uses different option types to choose different behaviors. Using incorrect option version usually results in errors.
The latest one may continue to work for future server versions, but there is no guarantee.
Version | Type |
---|---|
1.16 | ScrcpyOptions1_16 |
1.17 | ScrcpyOptions1_17 |
1.18~1.20 | ScrcpyOptions1_18 |
1.21 | ScrcpyOptions1_21 |
1.22 | ScrcpyOptions1_22 |
1.23 | ScrcpyOptions1_23 |
1.24 | ScrcpyOptions1_24 |
1.25 | ScrcpyOptions1_25 |
2.0 | ScrcpyOptions2_0 |
2.1 | ScrcpyOptions2_1 |
This packets operates on Web Streams API streams.
NOTE: Web Streams API streams usually can't pipe between different implementations. This package by default uses the native implementation on globalThis
, and web-streams-polyfill
otherwise. If you are using another implementation, or simply not sure, wrap your streams using the WrapReadableStream
and WrapWritableStream
types from @yume-chan/stream-extra
package.
Requires a ReadableStream<Uint8Array>
that reads from the video socket.
import { ScrcpyOptions2_1, ScrcpyVideoStreamPacket } from "@yume-chan/scrcpy";
const options = new ScrcpyOptions2_1({
// use the same version and options when starting the server
});
const videoStream: ReadableStream<Uint8Array>; // get the stream yourself
// Parse video socket metadata
// Use `videoStream2` instead of `videoStream` from now on
const { metadata, stream: videoStream2 } =
await options.parseVideoStreamMetadata(videoStream);
const videoPacketStream: ReadableStream<ScrcpyMediaStreamPacket> =
videoStream2.pipeThrough(options.createMediaStreamTransformer());
// Read from `videoPacketStream`
Read audio stream is similar, but uses parseVideoStreamMetadata
instead.
Requires a WritableStream<Uint8Array>
that writes to the control socket.
Control socket is optional if control is not enabled. Video socket and control socket can run completely separately.
import {
ScrcpyControlMessageWriter,
ScrcpyOptions2_1,
} from "@yume-chan/scrcpy";
const options = new ScrcpyOptions2_1({
// use the same version and options when starting the server
});
const controlStream: ReadableWritablePair<Uint8Array, Uint8Array>; // get the stream yourself
const controlMessageWriter = new ScrcpyControlMessageWriter(
controlStream.writable.getWriter(),
options
);
// Call methods on `controlMessageWriter`
controlMessageWriter.injectText("Hello World!");
Requires a ReadableStream<Uint8Array>
that reads from the control socket.
import { ScrcpyDeviceMessageDeserializeStream } from "@yume-chan/scrcpy";
const controlStream: ReadableWritablePair<Uint8Array, Uint8Array>; // get the stream yourself
const deviceMessageStream: ReadableStream<ScrcpyDeviceMessage> =
controlStream.readable.pipeThrough(
new ScrcpyDeviceMessageDeserializeStream()
);
In Web Streams API, pipes will block its upstream when downstream's queue is full (back-pressure mechanism). If multiple streams are separated from the same source (for example, all Scrcpy streams are from the same USB or TCP connection), blocking one stream means blocking all of them, so it's important to always read from all streams, even if you don't care about their data.
// if using `AdbScrcpyClient`
stdout
.pipeTo(
new WritableStream<string>({
write: (line) => {
// Handle or ignore the stdout line
},
})
)
.catch(() => {})
.then(() => {
// Handle server exit
});
videoPacketStream
.pipeTo(
new WritableStream<ScrcpyVideoStreamPacket>({
write: (packet) => {
// Handle or ignore the video packet
},
})
)
.catch(() => {});
deviceMessageStream
.pipeTo(
new WritableStream<ScrcpyDeviceMessage>({
write: (message) => {
// Handle or ignore the device message
},
})
)
.catch(() => {});
The data from createMediaStreamTransformer()
has two types: configuration
and data
.
export interface ScrcpyMediaStreamConfigurationPacket {
type: "configuration";
data: Uint8Array;
}
export interface ScrcpyMediaStreamDataPacket {
type: "data";
keyframe?: boolean;
pts?: bigint;
data: Uint8Array;
}
When sendFrameMeta: false
is set, there will be no configuration
packets, and no keyframe
and pts
fields in data
packets. It's commonly used with decoders that can parse the media stream itself like FFmpeg, or saved unprocessed.
Otherwise, both configuration
and data
packets are available.
-
configuration
packet will be the first packet, and contains codec information. It will occasionally be sent again if the stream configuration changes. -
pts
(andkeyframe
field from server version 1.23) fields indata
packets are available to help decode the stream.
@yume-chan/scrcpy-decoder-tinyh264
and @yume-chan/scrcpy-decoder-webcodecs
can be used to decode and render the video stream in browsers. Refer to their README files for compatibility and usage information.