Skip to content

RTMP acknowledge overflow causes the encoder to stop streaming #730

Closed

Description

@winlinvip

ACK SIZE refers to the number of bytes that the PEER should send an Acknowledge message after receiving. In other words, it indicates the number of bytes that the PEER has received.
RTMP is 32, which means it is 4 bytes long, but it does not explicitly state how overflow should be handled, whether it is int32 or uint32.

5.3. Acknowledgement (3)
   The client or the server sends the acknowledgment to the peer after
   receiving bytes equal to the window size. The window size is the
   maximum number of bytes that the sender sends without receiving
   acknowledgment from the receiver. The server sends the window size to
   the client after application connects. This message specifies the
   sequence number, which is the number of the bytes received so far.
0         1         2         3
01234567890123456789012345678901 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|    sequence number (4 byte)   | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 shows the payload for the protocol message 'Acknowledgement'.
   sequence number: 32 bits
     This field holds the number of bytes received so far.

Previously, the bitrate of videos was very low, such as 100kbps, so there would be no overflow issue with these 4 bytes. Now, the bitrate is 80 to 100 times higher, which easily leads to overflow. Some encoders have different rules for handling this overflow, causing them to receive incorrect values and mistakenly assume that the server has not acknowledged, resulting in re-streaming.

TRANS_BY_GPT3

Activity

  1. added this to the milestone on Jan 6, 2017
  2. winlinvip commented on Jan 6, 2017

    @winlinvip
    MemberAuthor

    Gao Sheng has encountered something similar, but did not explain it clearly: #598

    TRANS_BY_GPT3

  3. winlinvip commented on Jan 6, 2017

    @winlinvip
    MemberAuthor

    Play a stream with a bitrate of 8Mbps using Flash player, set the ack size to 10*1024*1024, and print out the acknowledge messages sent by the Flash player.

    `ack=1` indicates an acknowledge message.
    `seq` is the sequence number, which represents the number of bytes received by the client.
    `sent` is the number of bytes displayed as transmitted by the server.
    
    [2017-01-06 11:23:30.321][warn][13607][114][35] ack=1, seq=0x500ac2, sent=0x5766fb, diff=482361
    [2017-01-06 11:23:35.493][warn][13607][114][35] ack=1, seq=0xa0192a, sent=0xa0f73a, diff=56848
    [2017-01-06 11:23:39.153][warn][13607][114][35] ack=1, seq=0xf0229e, sent=0xf4e6bb, diff=312349
    [2017-01-06 11:23:43.197][warn][13607][114][35] ack=1, seq=0x140241e, sent=0x140499e, diff=9600
    

    The ack size is set to 10*1024*1024, which means an acknowledge message should be sent after the player has played 10MB of data.
    For a flash player, the interval between sending messages, which is the difference in seq, is 0xa0192a-0x500ac2=5246568=5*1024*1024, which means it sends approximately half of the ack size.
    Subsequently, it sends acknowledgements after receiving 5.002307891845703MB and 5.0003662109375MB, which is roughly half of the window size.

    The following messages are:

    [2017-01-06 11:50:16.010][warn][13607][114][35] ack=1, seq=0x68ba6930, sent=0x68c14790, diff=450144
    [2017-01-06 11:50:20.063][warn][13607][114][35] ack=1, seq=0x690a7297, sent=0x690d58ae, diff=189975
    [2017-01-06 11:50:23.727][warn][13607][114][35] ack=1, seq=0x695a79a2, sent=0x695dafa8, diff=210438
    [2017-01-06 11:50:28.507][warn][13607][114][35] ack=1, seq=0x69aa7fe2, sent=0x69b1d0f1, diff=479503
    

    The respective values are: 5.00229549407959, 5.0017194747924805, 5.00152587890625.

    Therefore, it can be confirmed that the behavior of the flash player is to send an acknowledge confirmation message to the server when it receives more than 1/2 of the ack size (window size).

    TRANS_BY_GPT3

  4. winlinvip commented on Jan 6, 2017

    @winlinvip
    MemberAuthor

    Regarding the server side, my colleague has tested two of them:

    FMS: 0xee77d0a7
    NGINX-RTMP: 0xf0000000
    

    Once these two thresholds are exceeded, the server will reset the counter to 0.

    NGINX-RTMP code: https://github.com/arut/nginx-rtmp-module/blob/c0bf381d10de05c135f913921c58272838d5e1ee/ngx_rtmp_handler.c#L275

                if (s->in_bytes >= 0xf0000000) {
                    ngx_log_debug0(NGX_LOG_DEBUG_RTMP, c->log, 0,
                                   "resetting byte counter");
                    s->in_bytes = 0;
                    s->in_last_ack = 0;
                }
    

    This number has no basis and there is no comment explaining its source.
    In nginx-rtmp, the ack is sent when it exceeds the ack window (unlike Flash, which sends it when it exceeds half the window).

                if (s->ack_size && s->in_bytes - s->in_last_ack >= s->ack_size) {
    
                    s->in_last_ack = s->in_bytes;
    
                    ngx_log_debug1(NGX_LOG_DEBUG_RTMP, c->log, 0,
                            "sending RTMP ACK(%uD)", s->in_bytes);
    
                    if (ngx_rtmp_send_ack(s, s->in_bytes)) {
                        ngx_rtmp_finalize_session(s);
                        return;
                    }
                }
    

    TRANS_BY_GPT3

  5. winlinvip commented on Jan 6, 2017

    @winlinvip
    MemberAuthor

    About the handling of flash player overflow, the log is as follows:

    [2017-01-06 12:24:16.129][warn][13607][114][35] ack=1, seq=0xed47b9cd, sent=0xed49da95, diff=139464
    [2017-01-06 12:24:20.549][warn][13607][114][35] ack=1, seq=0xed97c0e6, sent=0xed992be4, diff=92926
    [2017-01-06 12:24:26.044][warn][13607][114][35] ack=1, seq=0xede7c793, sent=0xeded7128, diff=371093
    [2017-01-06 12:24:29.706][warn][13607][114][35] ack=1, seq=0xee37cff6, sent=0xee3d82a9, diff=373427
    [2017-01-06 12:24:34.113][warn][13607][114][35] ack=1, seq=0xee87dca3, sent=0xee90a823, diff=576384
    [2017-01-06 12:24:37.781][warn][13607][114][35] ack=1, seq=0xeed7e36a, sent=0xeedd0cbf, diff=338261
    [2017-01-06 12:24:42.201][warn][13607][114][35] ack=1, seq=0xef27ed1a, sent=0xef2e1b6d, diff=405075
    [2017-01-06 12:24:46.589][warn][13607][114][35] ack=1, seq=0xef77f9ad, sent=0xef7d5424, diff=350839
    [2017-01-06 12:24:51.016][warn][13607][114][35] ack=1, seq=0xefc806b8, sent=0xefc8896a, diff=33458
    [2017-01-06 12:24:55.432][warn][13607][114][35] ack=1, seq=0x500eb7, sent=0xf01a1a8a, diff=-271971373
    [2017-01-06 12:25:00.178][warn][13607][114][35] ack=1, seq=0xa01113, sent=0xf06da792, diff=-271739265
    [2017-01-06 12:25:04.577][warn][13607][114][35] ack=1, seq=0xf0186e, sent=0xf0bc6d49, diff=-271821605
    [2017-01-06 12:25:09.373][warn][13607][114][35] ack=1, seq=0x140210b, sent=0xf10fe9e5, diff=-271595302
    [2017-01-06 12:25:12.673][warn][13607][114][35] ack=1, seq=0x19021c6, sent=0xf15a736a, diff=-271953500
    [2017-01-06 12:25:17.435][warn][13607][114][35] ack=1, seq=0x1e0287d, sent=0xf1aa3d1b, diff=-271969122
    [2017-01-06 12:25:22.213][warn][13607][114][35] ack=1, seq=0x2302f34, sent=0xf1f9d01b, diff=-271998745
    [2017-01-06 12:25:26.996][warn][13607][114][35] ack=1, seq=0x2803a9b, sent=0xf24c3340, diff=-271845211
    [2017-01-06 12:25:31.382][warn][13607][114][35] ack=1, seq=0x2d03ed4, sent=0xf299ecd1, diff=-271995395
    [2017-01-06 12:25:36.534][warn][13607][114][35] ack=1, seq=0x3204cee, sent=0xf2f33cf8, diff=-271388662
    [2017-01-06 12:25:40.566][warn][13607][114][35] ack=1, seq=0x37053d6, sent=0xf33de12b, diff=-271741611
    [2017-01-06 12:25:45.332][warn][13607][114][35] ack=1, seq=0x3c05a44, sent=0xf38b265b, diff=-271922153
    

    The overflowing data is padded with either 32 bits or 31 bits, which is significantly different from what flash sends. If we directly set it to zero, it can match, just like in nginx-rtmp, where it is set to zero when it exceeds 0xf0000000.

    f3c262d3-099a-4526-a19e-6167ccb03bbb

    TRANS_BY_GPT3

  6. winlinvip commented on Jan 6, 2017

    @winlinvip
    MemberAuthor

    This problem has only two solutions:

    1. If the accumulation of acknowledge exceeds 0xf0000000, it is set to zero.
    2. Send an acknowledge message if it exceeds half of the ack size.

    Supported in version 2.0.225.

    TRANS_BY_GPT3

  7. added 4 commits that reference this issue on Jan 6, 2017
  8. winlinvip commented on Jan 6, 2017

    @winlinvip
    MemberAuthor

    In addition, some encoders do not send an ack size (usually indicating that they do not need to send an acknowledge confirmation message), but they still want the server to send a confirmation message. Therefore, it is necessary to support configuration that can change the ack size for both input and output.

    Supported in 3.0.13.

    TRANS_BY_GPT3

  9. added a commit that references this issue on Jan 6, 2017
  10. added 2 commits that reference this issue on Feb 25, 2017
  11. self-assigned this
    on Sep 18, 2021
  12. changed the title [-]RTMP acknowledge 溢出导致编码器停止推流[/-] [+]RTMP acknowledge overflow causes the encoder to stop streaming[/+] on Jul 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

BugIt might be a bug.TransByAITranslated by AI/GPT.

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions