Add Tuya LAN transport: local two-way audio without cloud credentials (tuya-lan: source) - #2511
Open
RomuloGatto wants to merge 5 commits into
Open
RomuloGatto wants to merge 5 commits into
RomuloGatto wants to merge 5 commits into
Conversation
Tuya cameras speak their signaling protocol on the local network (TCP 6668) as
well as through the cloud. That local channel is authenticated with the device's
16-byte local key and reproduces what the mobile apps do: an offer/answer/candidate
exchange that ends in the same Pion WebRTC session, so the existing media path is
reused and only the transport differs.
This adds a second transport next to the existing Smart/Cloud API ones:
streams:
camera:
- rtsp://user:pass@192.168.1.10:554/live/ch0 # video + camera microphone
- tuya-lan://192.168.1.10?device_id=XXX&local_key=XXXXXXXXXXXXXXXX
No Tuya account, cloud project or internet access is involved at runtime - the
local key is read from the query string or from a JSON file
(`tuya-lan:/etc/go2rtc/tuya-lan.json`), which keeps it out of the config.
Implementation:
- pkg/tuya/lan.go: the 3.4 protocol (AES-ECB/PKCS7 payloads, HMAC-SHA256 frames,
framing over a TCP connection) plus the signaling client. The WebRTC answer is
normalized: cameras that tunnel video over Tuya's private KCP section get that
section replaced with an inactive placeholder, so Pion can negotiate the
bidirectional G.711 audio track.
- The signal client is now an interface (GetSignal/SetHandlers) so the MQTT and
LAN transports share the client code, and Protocol 312 (speaker) is sent on the
LAN transport, where the camera acknowledges it.
- The LAN audio is forwarded as received (G.711 20 ms frames): the cloud path's
240-byte reframing only exists to cut delay behind the Tuya relay and makes this
camera drop the audio completely.
- G.711/8000 is advertised for the LAN microphone track - browsers only offer
G.711/Opus, and Tuya reports the camera audio as codecType 101 (PCML).
Also fixes two non-constant format string vet warnings in cloud_api.go, without
which `go test ./pkg/tuya` cannot run.
Unit tests for the parts that do not need a camera: 3.4 frame encoding/decoding (including tampering and wrong key), answer normalization for cameras that tunnel video through Tuya's KCP section, source parsing from the query string and from a JSON file (defaults and validation), and the G.711 codec list used for the microphone track. The README gains a `Tuya LAN API` section with both config forms, the parameters and the two caveats found on real hardware: cameras that can only be reached across a subnet need an ICE server (`stun=`), and most cameras hold a single audio session, so the LAN session can silence the camera's RTSP audio.
go vet (and therefore the default `go test`) refuses to build the package with fmt.Errorf(dynamic). Use errors.New for the API message, no behaviour change.
The camera hands out only a couple of concurrent media sessions and keeps each one occupied for minutes after the client goes away. When a session is refused the handshake, offer and ICE negotiation still complete - the camera just never sends media - and a producer whose Start() fails is retried immediately (its retry counter restarts at zero), so the retry loop itself keeps the camera saturated and the source stays dead until go2rtc is restarted. - fail the dial (not Start) when a completed session carries no media - bound handshake+offer+ICE so a saturated camera cannot leave Dial blocked forever - space failed sessions out (2s doubling, capped at 2 min) and reject further dials without touching the camera while the window is active
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a second transport to the Tuya integration: the camera's local signaling protocol
(TCP 6668, authenticated with the 16-byte device local key), next to the existing Smart API and
Cloud API paths. Media still flows over the same Pion WebRTC session; only the signaling
transport differs, so this is a new
tuya-lan:source rather than a second media implementation.The intended source layout separates the two audio directions explicitly:
All LAN parameters can instead live in a JSON file, keeping the local key out of the go2rtc
configuration:
Why
local key is enough for standard cameras. Cameras that validate the app handshake also need
the static WebRTC session metadata that the apps fetch once from the cloud.
against (VDS CIPTZW-3M PTZ, Tuya-based Happytimesoft firmware) never played outgoing audio
through the cloud path; over the LAN the speaker works.
tuya-lan:supplies the talkback direction used by a browser/Frigate microphone or another go2rtc
consumer.
How it works
pkg/tuya/lan.goimplements protocol 3.4: AES-ECB/PKCS7 payloads, HMAC-SHA256 frames, the0x000055AAframing over TCP, and the signaling client (offer, answer, candidate and Protocol312 speaker command). Only protocol 3.4 is implemented; this camera rejects 3.5.
Dialtime (tuya-lan:-> local TCP;otherwise the existing Smart/Cloud paths). A small interface (
GetSignal/SetHandlers)lets all three transports share the existing WebRTC media client.
section (
m=application 9 tuya): that unusable section is replaced with an inactive videoplaceholder so Pion can negotiate the bidirectional G.711 audio track. Answers with a regular
H264 section pass through unchanged.
failure is returned instead of being discarded in a detached goroutine.
to reduce relay delay and made this camera drop audio; LAN keeps the browser's standard 20 ms
G.711 frames.
this camera's audio as codec type 101 (mapped to
PCMLby the cloud dialect).retries use a per-device exponential backoff (2 seconds to 2 minutes), allowing that camera's
small session table to drain without blocking other cameras. Smart and Cloud transports retain
their previous setup behavior.
Verified on hardware
Camera: VDS CIPTZW-3M PTZ (Tuya / Happytimesoft), with the go2rtc host on another VLAN.
protocol=3.4 handshake_response cmd=4 status=0 payload_len=48streamType=2, answerm=audio 9 UDP/TLS/RTP/SAVPF 0/a=rtpmap:0 PCMU/8000, Protocol 312 speaker command accepted by the cameraindependent recording of the camera microphone contains the pattern with peak/quiet band-power
ratios of approximately
3e3and4e5in two runs, proving that the speaker played itconsumer; the human in the room confirmed the phrase was audible repeatedly
Tests
The package tests cover protocol 3.4 framing and authentication failures, LAN answer
normalization, URL/file configuration, G.711 codec negotiation, retry bounds and retry isolation
between cameras.
go test ./...also reaches the Tuya package successfully. The repository currently has unrelatedbaseline failures on the PR base (platform-dependent FFmpeg snapshots, stale/missing symbols and
existing vet findings); the same failing package set reproduces on base commit
c245815.Notes / limitations
stun=stun:192.168.100.5:3478. Without it, signaling can succeed while ICE never carries media.window. The per-device backoff prevents failed retries from keeping that session table full.
and camera audio, and use
tuya-lan:as the talkback source.above. Other Tuya firmware families may use a different SDP layout or signaling dialect.
Related issues: #2478 (outgoing audio delivered but never played, Protocol 312 unanswered) and
#2467 (audio negotiated as PCML/8000 without a receiver) are cloud-path symptoms. This PR leaves
those paths unchanged; its findings are consistent with them: the speaker command is accepted on
the LAN channel and the camera's audio is G.711.