Why does Block Transfer (Get-Request-Next) for Profile Generic (Class 7) buffer fail with block-number-out-of-sequence over TCP 4059?

[elena_rostova]

[!WARNING]
Technical Redaction & Safe Harbor Notice:
All communication captures, system titles, and cryptographic parameters in this thread have been sanitized and neutralized. Real utility meter identifiers, production encryption keys (GUEK/GAK/MK), SIM IMSIs, and proprietary network envelopes have been substituted with standard IEC 62056 synthetic test vectors. Do not post unredacted production captures or live cryptographic keys.

We are testing automated bulk readout of 15-minute Load Profile Generic (Class 7, OBIS 1.0.99.1.0.255, Attribute 2 buffer) on polyphase smart meters running firmware build v3.1.2 over IEC 62056-47 TCP wrapper (port 4059). The association is established under Management Client (Client SAP 0x0001, Logical Device 1) with Security Suite 1 (General-Glo-Ciphering 0xDB with AES-GCM-128, HLS 5 GMAC).

When we request a 15-day historical slice (approximately 420 KB, requiring ~420 datablocks at 1024 bytes/block), the transfer streams cleanly from block #1 up to block #13. However, immediately upon receiving the HES Get-Request-Next for block #14, the meter terminates the stream with a service error response (0xC4 0x01 C1 01 02: block-number-out-of-sequence).

Here is the initial Get-Request-Normal for the Profile Generic buffer:

00 01 00 01 00 01 00 28 DB 08 53 48 41 00 00 00 00 01 30 00 00 00 10 4A 1B 88 E2 91 04 FF 12 33 AA 01 C0 01 C1 00 07 01 00 63 01 00 FF 02 00

And here is the failing exchange at block #14:

• HES Get-Request-Next (Block #14):

00 01 00 01 00 01 00 1C DB 08 53 48 41 00 00 00 00 01 30 00 00 00 23 88 1F 3C 90 22 EE 45 10 99 AA 01 C0 02 C1 00 00 00 0E

• Meter Response:

00 01 00 01 00 01 00 17 DB 08 53 48 41 00 00 00 00 01 30 00 00 00 1F 77 CC 11 02 AA 90 12 44 88 01 C4 01 C1 01 02

Dissecting the plain APDU inside the ciphered envelope:

• C4 01: Get-Response-Normal (with service error)
• C1: Invoke-ID 1
• 01: Result code 1 (hardware-fault / exception)
• 02: Error detail 2 (block-number-out-of-sequence)

Our initial hypothesis was that the meter's internal RAM streaming buffer suffered a memory pool exhaustion under high TCP socket throughput. Has anyone observed block sequencing rejections at specific block thresholds?

[marcus_vance]

Elena, I disagree that this is an embedded RAM heap exhaustion. If the meter suffered memory exhaustion, it would typically reset the TCP socket with an RST flag or return data-access-result: temporary-failure (0x02), rather than returning a structured Get-Response-Normal with block-number-out-of-sequence.

Look closely at your HES Get-Request-Next APDU:
C0 02 C1 00 00 00 0E

• C0 02: Get-Request-Next tag
• C1: Invoke-ID-And-Priority (0xC1 = Invoke ID 1, Priority Normal)
• 00 00 00 0E: Block number 14 (Double-Long-Unsigned, 4 bytes)

Per DLMS Green Book 10th Ed Section 9.2.4 (DataBlock Services) and IEC 62056-53 Section 8.3.3, Get-Request-Next specifies:

Get-Request-Next ::= SEQUENCE {
    invoke-id-and-priority  Invoke-Id-And-Priority,
    block-number            Double-Long-Unsigned
}

Notice that in your meter's previous Get-Response-With-Datablock (Block #13), the meter returned block-number: 0x0000000D. If your HES Netty pipeline uses zero-based indexing or increments before acknowledgment, it could trigger this. But even more critically: is your HES maintaining an exact 1-to-1 lockstep request/response window, or is it pipelining Get-Request-Next calls?

[david_miller]

Marcus raises a valid point regarding the APDU definition, but looking at the Hex trace, Elena's HES is strictly waiting for each Get-Response-With-Datablock before sending Get-Request-Next. The block number 0x0000000E directly follows 0x0000000D.

However, there is a much more subtle failure mode rooted in the meter's internal task scheduling:

Per DLMS Blue Book Ed. 17 Section 4.3.7.2 and IEC 62056-62 Section 6.2.14, Interface Class 7 (Profile Generic) maintains a dynamic circular buffer. If the meter's internal real-time clock (Class 8, 0.0.1.0.0.255) triggers a scheduled capture event (such as the top-of-the-hour or 15-minute boundary tick capture_period: 900s) while the xDLMS server task is actively serializing datablocks, what happens?

If the firmware does not maintain an immutable snapshot row-pointer upon receiving the initial Get-Request-Normal (Block #1), the background measurement task executes Class 7 Method 2 (capture) in the middle of the transfer. This inserts a new row at the head of the buffer, shifting all internal row offsets by 1. When the server tries to fetch the next slice for Block #14, the internal buffer length check detects a row-pointer hash mismatch and raises block-number-out-of-sequence.

Elena, can you check your meter's RTC timestamp logs at the exact moment Block #14 failed?

[alex_turner]

David's point about asynchronous profile capture shifting row pointers is brilliant, but there is an even deeper cryptographic trap in Elena's captured trace that explains the exact failure.

Look at the Security Header and Invocation Counter of the failing Get-Request-Next:
DB 08 53 48 41 00 00 00 00 01 30 00 00 00 23 ...

• DB: General-Glo-Ciphering tag
• 08 53 48 41 00 00 00 00 01: System Title SHA00000001 (8 bytes)
• 30: Security Control byte (0x30 = Global Unicast Key, Authenticated + Encrypted with AES-GCM-128)
• 00 00 00 23: Invocation Counter ($IC = 35$)

Now, look at the previous block's request (Block #13): the HES sent $IC = 35$ (0x00000023) as well! Because of a transient TCP retransmission in the HES gateway layer, the HES re-used the exact same Invocation Counter 0x00000023 for Block #14.

Per DLMS Green Book 10th Ed Section 9.2.5.3 (Security Suite 1 Cryptographic Processes) and IEC 62056-53 Section 8.4.2:
$$\text{Rule: } IC_{\text{received}} > IC_{\text{last_authenticated}}$$

When the meter's AES-GCM engine received the duplicate $IC = 35$, it immediately aborted AES-GCM decryption because the counter failed the strictly monotonic anti-replay check. Instead of raising a fatal crypto exception, the Netty/COSEM handler fell through to the block sequencing state machine, throwing block-number-out-of-sequence!

[sarah_jenkins]

Alex has identified the root cryptographic trigger, and there is a secondary structural factor in the Profile Generic schema that exacerbates this when streaming over TCP.

Per DLMS Blue Book Ed. 17 Section 4.3.7.1 and IEC 62056-62 Table 18, Attribute 3 (capture_objects) of 1.0.99.1.0.255 defines the column schema of each row in the buffer. If your capture object list includes variable-length items (such as Octet-String status words, event log bitstrings, or extended tariffs) without using Compact-Array (0x13), the A-XDR encoder cannot calculate fixed-offset record boundaries:

Structure {
  class_id: 3 (Register), logical_name: 1.0.1.8.0.255, attribute_index: 2, // Active energy (+A)
  class_id: 3 (Register), logical_name: 1.0.2.8.0.255, attribute_index: 2, // Active energy (-A)
  class_id: 8 (Clock),    logical_name: 0.0.1.0.0.255, attribute_index: 2, // Date/Time (12B)
  class_id: 1 (Data),     logical_name: 0.0.96.11.0.255, attribute_index: 2 // Status word
}

When a 420 KB buffer is split into 1024-byte chunks, individual A-XDR structure tags (0x02 0x04) get sliced right across the 1024-byte boundary. If a single block retry or duplicate $IC$ occurs mid-stream, neither the HES nor the meter can resynchronize the BER stream offset without restarting from Block #1.

[elena_rostova]

Thank you all—this technical breakdown is spot on. We just analyzed the internal firmware logic and the RTOS execution traces, and both issues were occurring simultaneously:

1. RTC Capture Collisions: At exactly $T=15:00:00$, the 15-minute timer fired GXDLMSProfileGeneric.capture(), which inserted a new 36-byte row into the circular buffer. Because our firmware was referencing dynamic RAM pointers rather than freezing a read-only snapshot table during AARQ association, the remaining block count offset shifted by 1 row!
2. Invocation Counter Re-use: In our HES TCP communication log, the gateway experienced a 1200ms socket latency spike at Block #13, causing the client stack to re-send the request with the identical $IC = 35$. The meter's AES-GCM-128 engine rejected the frame due to $IC_{\text{rx}} \le IC_{\text{last}}$, leading to the service error.

Here is the exact internal verification from our test bench:

[DLMS-SECURITY] GMAC verification failed: Invocation Counter 0x00000023 <= Last 0x00000023
[DLMS-COSEM] Aborting Datablock Stream: expected block 14, received unauthenticated frame

[marcus_vance]

To solve this permanently from both the HES and Meter architecture sides, here is the dual-remediation strategy:

1. HES Pacing & Monotonic Counter Guarantee

In the HES communication pipeline, ensure that every physical socket transmission (including retries and Get-Request-Next frames) strictly invokes ciphering.invocation_counter += 1 before AES-GCM encryption. Retransmitting the exact same ciphertext byte stream is prohibited under Security Suite 1.

2. Selective Access by Date Range (IEC 62056-62 Section 6.2.14)

Instead of streaming the unconstrained 420 KB buffer using plain Get-Request-Normal, HES should always issue a Selective Access by Range query (Access Selector 1) specifying explicit from_value and to_value date/time bounds:

SelectiveAccessDescriptor ::= SEQUENCE {
  access_selector: 1, // Range Descriptor
  access_parameters: Structure {
    restricting_object: Structure { class_id: 8, logical_name: 0.0.1.0.0.255, attribute_index: 2 },
    from_value: 07 E8 08 01 FF 00 00 00 FF 80 00 00, // 2026-08-01 00:00:00
    to_value:   07 E8 08 0F FF 17 3B 3B FF 80 00 00, // 2026-08-15 23:59:59
    selected_values: Array [ ] // all columns
  }
}

This restricts the buffer dataset to a deterministic row count that is immune to background capture additions.

[david_miller]

From the protocol framing layer, there is another critical optimization: PDU Buffer Size Negotiation.

Per IEC 62056-53 Section 8.3.2 (Negotiated xDLMS Parameters) and DLMS Green Book Section 9.2.2, during the initial AARQ exchange (xDLMS-Initiate-Request), the client and server negotiate max-pdu-size:

InitiateRequest ::= SEQUENCE {
  proposed-dlms-version-number  Unsigned8 (6),
  proposed-conformance          ConformanceBitString,
  client-max-receive-pdu-size   Unsigned16 (2048) // Upgrade from 1024 to 2048
}

Over TCP/IP (port 4059), setting client-max-receive-pdu-size to 2048 or 4096 bytes cuts the total number of datablocks from 420 down to ~105 blocks. This reduces round-trip handshake overhead by 75% and significantly lowers the probability of colliding with a 15-minute RTC capture tick.

[alex_turner]

Here is the verified, working Get-Request-Next (Block #14) trace with strictly incremented Invocation Counter ($IC = 36 = 0x00000024$) and clean AES-GCM-128 authentication tag:

00 01 00 01 00 01 00 1C DB 08 53 48 41 00 00 00 00 01 30 00 00 00 24 5C 91 E3 7A 10 44 AA 99 21 BB 01 C0 02 C1 00 00 00 0E

And here is the meter's successful Get-Response-With-Datablock (Block #14):

00 01 00 01 00 01 04 0C DB 08 53 48 41 00 00 00 00 01 30 00 00 00 28 8A 19 BC 44 90 EF 33 11 02 C4 02 C1 00 00 00 00 0E 01 00 02 04 12 00 ...

Dissecting the plain response:

• C4 02: Get-Response-With-Datablock
• C1: Invoke-ID 1
• 00: last-block = false
• 00 00 00 0E: block-number = 14
• 01 00: Raw byte length of block payload
• 02 04: Structure with 4 elements (Active energy, reactive energy, date/time, status)

With $IC$ strictly incrementing on every attempt, the GMAC authentication succeeds instantaneously.

[elena_rostova]

To synthesize this for the community, here is the complete reference implementation in Python using Gurux.DLMS demonstrating how to read large Profile Generic buffers safely with Selective Access and monotonic Invocation Counter management:

from gurux_dlms.objects import GXDLMSProfileGeneric, GXDLMSClock
from gurux_dlms.enums import Security, AccessSelector
from datetime import datetime, timedelta

def read_profile_generic_safe(client, reader, obis_code="1.0.99.1.0.255"):
    pg = GXDLMSProfileGeneric(obis_code)
    
    # 1. Configure Selective Access by Date Range (Last 7 Days)
    clock = GXDLMSClock("0.0.1.0.0.255")
    end_time = datetime.utcnow()
    start_time = end_time - timedelta(days=7)
    
    # Restrict by Range Descriptor (Access Selector 1)
    access_params = [clock, start_time, end_time, []]
    
    # 2. Enforce Security Suite 1 with Monotonic IC Increment
    client.ciphering.security = Security.AUTHENTICATION_ENCRYPTION
    
    # 3. Read Rows using Block Transfer with guaranteed IC stepping
    print(f"[*] Reading Profile Generic {obis_code} via Selective Access...")
    cells = reader.readRowsByRange(pg, start_time, end_time)
    
    print(f"[+] Successfully retrieved {len(cells)} profile entries.")
    for row in cells[:3]: # Preview first 3 rows
        print(f"    Timestamp: {row[0]}, +A Import: {row[1]} Wh, Status: {hex(row[3])}")
    return cells

Summary Checklist for Engineers Encountering block-number-out-of-sequence:

1. Anti-Replay Counter Check: Verify that every retry/next block incremented the 4-byte $IC$ (IEC 62056-53 Section 8.4.2).
2. Buffer Snapshot Isolation: Ensure meter firmware copies row pointers to a read-only shadow buffer on AARQ (DLMS Blue Book Section 4.3.7.2).
3. Selective Access: Always query by explicit timestamp range rather than reading unbounded Attribute 2.
4. PDU Size Tuning: Negotiate max-pdu-size to $\ge 2048$ bytes to reduce datablock round trips.