Hii the issue is that L1 is crashing due to memory full
------------------------------------------------------------------------------------------------------------------------------------------------------
| MAC | MAC-to-PHY Tput | PHY-to-MAC Tput | UL FEC CB Iterations |
Cell (MU / DL,UL MHz) | Inst | kbps Num CB | kbps UL BLER Num CB | Min Avg Max | SRS SNR
-------------------------|------|------------------------|------------------------------------------------|-----------------------|--------------
0 (MU 1 / 70, 70) | 0 | 988,857 589,963 | 312 / 362 13.82% 2,001 | 1 2.07 12 | 0 Db
-------------------------------------------------------------------------------------------------------------------------------------------------------
Core Utilization [4 BBU core(s)]:
Core Id : 4 5 6 7 Avg
Numa Node : 0 0 0 0
Util % : 9.83 9.59 9.45 9.59 9.62
Intr % : 0.33 0.34 0.34 0.34 0.34
Spare % : 0.35 0.35 0.35 0.35 0.35
Sleep % : 89.47 89.70 89.84 89.70 89.68
Numa % : 0.00 0.00 0.00 0.00 0.00
TTI Cnt : 10002 10002 10002 10002
TTI Min : 0 0 0 0
TTI Avg : 9 9 8 9
TTI Max : 67 67 67 67
Active Cores : Min[ 0] Avg[ 0.00] Max[ 0]
Xran Id: 8 9 Master Core Util: 54 %
-------------------------------------------------------------------------------------------------------------------------------------------------------
…
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 458 18)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 0)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 1)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 458 19)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 2)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 0)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 3)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 1)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 4)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 2)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 5)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 3)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 6)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 4)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 7)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 5)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 8)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 6)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 9)
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 7)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 10)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 8)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 11)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 9)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 12)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 10)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 13)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 11)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
[err] file[nr5g/api/mac2phy/gnb_mac2phy_api_proc.c] func[nr5g_gnb_mac2phy_api_proc_get_new_list] line[588]: Stopping Phy because MAX_SUCCESSIVE_NO_API_TTIS_BEFORE_STOP: 15
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(DL): phyInstance[0] inc[1] (sfn: 459 14)
[err] file[auxlib/wls/aux_wls_mac2phy_api_recv.c] func[wls_mac2phy_api_recv_add_late_api] line[318]: Cleanup Queue is Full
[err] file[nr5g/api/mac2phy/gnb_mac2phy_api_proc.c] func[nr5g_gnb_mac2phy_api_proc_get_new_list] line[588]: Stopping Phy because MAX_SUCCESSIVE_NO_API_TTIS_BEFORE_STOP: 15
nr5g_gnb_mac2phy_api_error_check_use_empty_channels(UL): 0 1 (sfn: 459 12)
Stopping because: PHYDI_STATE_FORCE_STOP. gStop = 0
Timeout from stopping XRAN_timing_thread.. [10000]
Disable RU0 CC0 [0000000000000000:0]
Timing Thread is not running! 0
phydi_stop[from 8]: phyInstance: -1, sendStop: 0, phyIdStart: 0, phyIdStop: 40
MLogPrint: ext_filename((null).bin)
Opening MLog File: l1mlog_wls-c0.bin
MLog file l1mlog_wls-c0.bin closed
Mlog Print with Time successful
PHY_STOP PhyInstance[0] nPhyDiState[4] nPhyDiStartMode[4] nPhyDiStateCount[0]
XRAN Stop! RU0 [1]
Received total number of stops! Stopping......
fh_rx_bbdev-9 worker thread finished on core 9 [worker id 0]
EBBUPOOL [INFO][ebbu_pool_consumer_remove_all_cores]:Destroy the consumer thread 0 in core 4 successfully!
EBBUPOOL [INFO][ebbu_pool_consumer_remove_all_cores]:Destroy the consumer thread 1 in core 5 successfully!
EBBUPOOL [INFO][ebbu_pool_consumer_remove_all_cores]:Destroy the consumer thread 2 in core 6 successfully!
EBBUPOOL [INFO][ebbu_pool_consumer_remove_all_cores]:Destroy the consumer thread 3 in core 7 successfully!
EBBUPOOL [INFO][ebbu_pool_consumer_remove_all_cores]:All consumer threads are destroyed!
EBBUPOOL [INFO][ebbu_pool_maintenance]:Framework Maintenance thread is destroyed successfully!
EBBUPOOL [INFO][ebbu_pool_release_threads]:Threads Released successfully!
EBBUPOOL [INFO][ebbu_pool_release_queues]:Release queues successfully!
EBBUPOOL [INFO][ebbu_pool_release_handler]:Release handler successfully!
phydi_shutdown[from 8]: phyInstance: -1, sendStop: 0, testFileName: 0, phyIdStart: 0, phyIdStop: 40
PHY_SHUTDOWN PhyInstance[0] nPhyDiState[32] nPhyDiStateCount[0]
XRAN Close RU0 [0:00000000]
mem_mgr_display_size:
Num Memory Alloc: 20
Total Memory Size: 183,731,456
wls_print_stats
nWlsInst[0] nCellMapping[0x0000010000ffffff]
Cell QueueAlloc CellAlloc CellSend Diff1 Diff
0 18202938 18202842 18202842 96 0
1 96 0 0 96 0
2 96 0 0 96 0
3 96 0 0 96 0
4 96 0 0 96 0
5 96 0 0 96 0
6 96 0 0 96 0
7 96 0 0 96 0
8 96 0 0 96 0
9 96 0 0 96 0
10 96 0 0 96 0
11 96 0 0 96 0
12 96 0 0 96 0
13 96 0 0 96 0
14 96 0 0 96 0
15 96 0 0 96 0
16 96 0 0 96 0
17 96 0 0 96 0
18 96 0 0 96 0
19 96 0 0 96 0
20 96 0 0 96 0
21 96 0 0 96 0
22 96 0 0 96 0
23 96 0 0 96 0
40 22085730 22085635 22085635 95 0
xran_timing_source_thread:2265[err] poll_next_tick too long, delta:2982720262(ns), tUsed:4463677606(tick)
Closing timing source thread...
PHY>
PHY>
PHY>
PHY>
PHY>
PHY>stopphy
begin stopping phy (0)...Timing Thread is not running! -1
PHY>application termination...phy_print_thread: [PID: 2362807]... Stopping
BBDEV Stats: Enqueued (1256308418) | Dequeued (1256308418) | Enqueue Errors (0) | Dequeue Errors (0)
BBDEV Queue's Stopped for device 0
wls_nrt_mac2phy_api_recv_thread: [PID: 2362806]... Stopping
wls_layer_close:WLS interface termination...
wls_stop_rx_thread:Termination WLS Receiver thread...
wls_layer_close:WLS Free ....
wls_layer_close:WLS Close ....
wls_lib: WLS_Close
wls_lib: un-link: 0 <-> 1
WLS_Close(0): nWlsClients[1]
wls_layer_close:Done
generate tracelog file [./phy_tracelog_wls.log] in L1APP directory successfully
Cleanup after [PID] 2362622
what can be possible reasons and how to resolve ?