[Feature] Implement DMA support by BenkangPeng · Pull Request #293 · tancheng/VectorCGRA

BenkangPeng · 2026-06-02T13:55:27Z

This PR introduces CgraDmaRTL which integrates the CGRA with a DMA engine, enabling direct memory transfers between external DRAM(don't implement now) and the CGRA's dataSPM.

tancheng · 2026-06-02T21:31:46Z

+    s.mem_rd_req_val    = OutPort() # dma_read_request_valid
+    s.mem_rd_req_rdy    = InPort() # dma_read_request_ready
+    s.mem_rd_req_addr   = OutPort(DmaDramAddrType)


Why can't we use the RecvIfcRTL and SendIfcRTL interfaces to connect the DmaRTL?

tancheng · 2026-06-03T18:58:19Z

+      s.data_mem.spm_dma_rval       //= s.spm_dma_rval
+      s.data_mem.spm_dma_rrdy       //= s.spm_dma_rrdy
+      s.data_mem.spm_dma_raddr      //= s.spm_dma_raddr
+      s.data_mem.spm_dma_rresp_val  //= s.spm_dma_rresp_val
+      s.data_mem.spm_dma_rresp_rdy  //= s.spm_dma_rresp_rdy
+      s.data_mem.spm_dma_rresp_data //= s.spm_dma_rresp_data


As we discussed before, should we connect the dma to the controller (as intermediate interface/transition)? instead of directly connecting to data spm?

So then we can leverage

VectorCGRA/controller/ControllerRTL.py

Lines 138 to 139 in 44618d5

s.recv_from_cpu_pkt //= s.recv_from_cpu_pkt_queue.recv

s.send_to_cpu_pkt //= s.send_to_cpu_pkt_queue.send

to decide the next location (e.g., local spam bank, remote spm)?

…ests.

…RecvIfcRTL. Replace `mem` with `dram` for clarity.

HobbitQia · 2026-06-04T15:05:31Z

Hi @tancheng @BenkangPeng , I summarized two direction of DMA design as below:

Rely on data controller
- DMA is added as a new client of the DataMemControllerRTL, where data in the DMA engine communicates directly with the DataMemControllerRTL, and the logic for multiplexing SPM ports is also implemented in that module.
  
  To initiate DMA, the CPU can send dma_mvin or dma_mvout to the CGRA, after which the controller activates the DMA engine by sending start signals.
- Pros: Keeps the controller clean; provides a faster path because data does not go through the controller.
- Cons: Additional logic is required to feed DMA results into the control memory.
All in controller
- All decoding logic is handled in the controller. The logic for handling the logic port should still reside in the DataMemControllerRTL, since the data memory should have its own port multiplexing logic.
  
  The logic of packeting should also be implemented in the controller module.
- Pros: Unifies control and data memory within the controller (the controller is already connected to both control and data memory).
- Cons: Introduces complex control logic in the controller; results in a slower path.

I prefer the second method but I think there are still some logic should be written in DataMemControllerRTL. WDTY?

tancheng · 2026-06-05T08:30:27Z

Hi @HobbitQia, option 2 looks good to me. Though I am not sure what logic should be additionally in DataMemController?

HobbitQia · 2026-06-07T14:56:54Z

Hi @HobbitQia, option 2 looks good to me. Though I am not sure what logic should be additionally in DataMemController?

I am thinking if we want enable the concurrent running of DMA and traditional load/store, then we need to multiplex the port of Data SPM and I think this logic can be implemented in DataMemController. However, we can also transform the data from DRAM into packets and use the command like CMD_STORE_REQUEST or CMD_LOAD_REQUEST to send to SPM. Then we can entirely write our logic in controller but with higher latency. I think maybe the former method can have better performance with minimal addition to DataMemController.

tancheng · 2026-06-07T18:15:57Z

Hi @HobbitQia, option 2 looks good to me. Though I am not sure what logic should be additionally in DataMemController?

I am thinking if we want enable the concurrent running of DMA and traditional load/store, then we need to multiplex the port of Data SPM and I think this logic can be implemented in DataMemController. However, we can also transform the data from DRAM into packets and use the command like CMD_STORE_REQUEST or CMD_LOAD_REQUEST to send to SPM. Then we can entirely write our logic in controller but with higher latency. I think maybe the former method can have better performance with minimal addition to DataMemController.

I thought they are the same latency if we can distinguish the CMD_STORE_REQUEST into CMD_STORE_REQUEST_FROM_NOC and CMD_STORE_REQUEST_FROM_CPU (and add another inport on the xbar) in controller?

Adding logic inside the DataMemController kind of bypassing the CGRA controller, which doesn't align with your Option 2, WDYT?

HobbitQia · 2026-06-08T02:15:57Z

Hi @HobbitQia, option 2 looks good to me. Though I am not sure what logic should be additionally in DataMemController?

I am thinking if we want enable the concurrent running of DMA and traditional load/store, then we need to multiplex the port of Data SPM and I think this logic can be implemented in DataMemController. However, we can also transform the data from DRAM into packets and use the command like CMD_STORE_REQUEST or CMD_LOAD_REQUEST to send to SPM. Then we can entirely write our logic in controller but with higher latency. I think maybe the former method can have better performance with minimal addition to DataMemController.

I thought they are the same latency if we can distinguish the CMD_STORE_REQUEST into CMD_STORE_REQUEST_FROM_NOC and CMD_STORE_REQUEST_FROM_CPU (and add another inport on the xbar) in controller?

Adding logic inside the DataMemController kind of bypassing the CGRA controller, which doesn't align with your Option 2, WDYT?

If the DMA data should go through the controller packet path, there may be extra latency of packeting, and there may be competitions between NoC/CPU/tile request to SPM? Or we have two separate paths in Controller?

tancheng · 2026-06-08T04:21:57Z

there may be extra latency of packeting

Packing can just be combinational logic before putting into a queue.

and there may be competitions between NoC/CPU/tile request to SPM? Or we have two separate paths in Controller?

I was thinking about separate paths, so mentioned FROM_CPU and FROM_NOC. I feel the requests targeting same SPM bank should anyway conflict with each other, WDYT?

HobbitQia · 2026-06-08T04:41:47Z

there may be extra latency of packeting

Packing can just be combinational logic before putting into a queue.

and there may be competitions between NoC/CPU/tile request to SPM? Or we have two separate paths in Controller?

I was thinking about separate paths, so mentioned FROM_CPU and FROM_NOC. I feel the requests targeting same SPM bank should anyway conflict with each other, WDYT?

Got it. I mean the conflict between different paths. Even we have FROM_CPU and FROM_NOC, there may be concurrent read/write on SPMs. So maybe we need to multiplex the in/out ports of SPMs and the logic of multiplexing and handling the conflicts can be implemented in DataSPMController?

tancheng · 2026-06-08T05:21:11Z

there may be extra latency of packeting

Packing can just be combinational logic before putting into a queue.

and there may be competitions between NoC/CPU/tile request to SPM? Or we have two separate paths in Controller?

I was thinking about separate paths, so mentioned FROM_CPU and FROM_NOC. I feel the requests targeting same SPM bank should anyway conflict with each other, WDYT?

Got it. I mean the conflict between different paths. Even we have FROM_CPU and FROM_NOC, there may be concurrent read/write on SPMs. So maybe we need to multiplex the in/out ports of SPMs and the logic of multiplexing and handling the conflicts can be implemented in DataSPMController?

Oh, we don't need to distinguish FROM_CPU and FROM_NOC, we can decompose the requests from CPU like:

VectorCGRA/controller/ControllerRTL.py

Line 207 in eb71842

    
           s.recv_from_cpu_pkt_queue.send.rdy @= s.crossbar.recv[kFromCpuCtrlAndDataIdx].rdy

HobbitQia · 2026-06-08T06:30:39Z

there may be extra latency of packeting

Packing can just be combinational logic before putting into a queue.

and there may be competitions between NoC/CPU/tile request to SPM? Or we have two separate paths in Controller?

I was thinking about separate paths, so mentioned FROM_CPU and FROM_NOC. I feel the requests targeting same SPM bank should anyway conflict with each other, WDYT?

Got it. I mean the conflict between different paths. Even we have FROM_CPU and FROM_NOC, there may be concurrent read/write on SPMs. So maybe we need to multiplex the in/out ports of SPMs and the logic of multiplexing and handling the conflicts can be implemented in DataSPMController?

Oh, we don't need to distinguish FROM_CPU and FROM_NOC, we can decompose the requests from CPU like:

VectorCGRA/controller/ControllerRTL.py

Line 207 in eb71842

s.recv_from_cpu_pkt_queue.send.rdy @= s.crossbar.recv[kFromCpuCtrlAndDataIdx].rdy

But if we add DMA, there should be another path FROM_DMA, which is different from FROM_CPU/FROM_NOC?

tancheng · 2026-06-08T17:12:23Z

But if we add DMA, there should be another path FROM_DMA, which is different from FROM_CPU/FROM_NOC?

Ah, yes, makes sense. That FROM_DMA would be similar to FROM_CPU. The DMA_Controller then can assemble the different ports into our struct and send/recv interface.

HobbitQia · 2026-06-09T01:08:28Z

DMA_Controller

So this DMA_Controller refers to DataSPMController or our DMA engine?

tancheng · 2026-06-09T01:28:47Z

So this DMA_Controller refers to DataSPMController or our DMA engine?

DMA engine in your figure, or @BenkangPeng's DmaEngineRTL.

tancheng · 2026-06-10T06:49:37Z

@BenkangPeng will you update this PR accordingly?

BenkangPeng · 2026-06-10T11:05:01Z

@BenkangPeng will you update this PR accordingly?

Yes, sorry for the delay. I will update this PR as soon as possible.

…RTL. The DMA is connected to the data memory controller indirectly via the controller, with the decoding logic integrated into the controller.

…lateRTL, and ControllerRTL.

… error of pytml verilog backend.

…interface for enhanced data transfer capabilities.

… then drives types from them

…te requests and adjust related signal handling for clarity and consistency.

…TL by passing DmaDataType and DmaCmdType as parameters, and updating related type definitions for improved clarity and consistency.

…rite requests, enhancing type definitions for DmaCmdType and DmaDataType

tancheng · 2026-06-15T01:45:32Z

    YType = mk_bits(max(clog2(multi_cgra_rows), 1))
    TileIdType = mk_bits(clog2(num_tiles + 1))
    ControllerXbarPktType = mk_controller_noc_xbar_pkt(InterCgraPktType)
+    DmaOpcodeType = DmaCmdType.get_field_type('opcode')


Similar to above attr, we should use kAttrOpcode instead of opcode here?

tancheng · 2026-06-15T01:45:46Z

    TileIdType = mk_bits(clog2(num_tiles + 1))
    ControllerXbarPktType = mk_controller_noc_xbar_pkt(InterCgraPktType)
+    DmaOpcodeType = DmaCmdType.get_field_type('opcode')
+    DmaDramAddrType = DmaCmdType.get_field_type('dram_addr')


ditto for dram_addr?

tancheng · 2026-06-15T01:47:12Z

+      s.dma_done //= s.controller.dma_done
+
+      # DMA engine <-> controller side of the SPM path.
+      s.dma_spm //= s.controller.dma_spm_from_dma


Rename this? dma_spm and dma_spm_from_dma sound confusing.

How about s.dma_spm_from_dma //= s.controller.dma_spm_from_dma? My original idea was that dma_spm indicates the signal is used for communication between DMA and SPM, with suffixes like from_dma and to_mem indicating the direction of the signal.

We should have two interfaces, one send and one recv, WDYT? and the "send"/"recv" should be in the name.

…Type

…proved clarity

tancheng · 2026-06-15T05:03:51Z

+
+    # Abstract external dram memory interfaces for the internal DMA engine.
+
+    s.dram_rd_req = SendIfcRTL(DmaDramAddrType)


Name this as s.send_dram_rd_req, to indicate its direction.

tancheng · 2026-06-15T05:04:04Z

+    # Abstract external dram memory interfaces for the internal DMA engine.
+
+    s.dram_rd_req = SendIfcRTL(DmaDramAddrType)
+    s.dram_rd_resp = RecvIfcRTL(DmaMemDataType)


name this as s.recv_dram_rd_resp.

tancheng · 2026-06-15T05:05:16Z

+
+    # Components.
+
+    s.cgra = CgraTemplateRTL(CgraPayloadType,


I was actually wondering why we need this CgraDmaRTL.py. Can we just expand the CgraTemplateRTL to include the DMA-related ports/interfaces?

tancheng · 2026-06-15T05:23:13Z

+    s.msg = OutPort( Type )
+    s.val = OutPort()
+    s.rdy = OutPort()
+    s.trace_len = len(str(Type()))


Can we just reuse the ValRdySendIfcRTL, but embed the trace_len into msg? The msg could have both msg.data and msg.trace_len. WDYT?

tancheng · 2026-06-15T05:24:57Z

+    - write   : Output (Send). DMA sends write requests to SPM.
+    - read    : Output (Send). DMA sends read requests to SPM.
+    - read_resp: Input  (Recv). DMA receives read data from SPM.


Can we reuse our send/recv interfaces, instead of creating new one?

tancheng · 2026-06-15T05:25:05Z

+    - write   : Input  (Recv). SPM receives write requests from DMA.
+    - read    : Input  (Recv). SPM receives read requests from DMA.
+    - read_resp: Output (Send). SPM sends read data back to DMA.


tancheng · 2026-06-15T05:25:50Z

+STATE_IDLE          = StateType( 0 ) # Waiting for a new DMA command
+STATE_MVIN_REQ      = StateType( 1 ) # MVIN: Issuing DRAM read request
+STATE_MVIN_RESP     = StateType( 2 ) # MVIN: Waiting for DRAM read response
+STATE_MVIN_WRITE    = StateType( 3 ) # MVIN: Writing unpacked words to SPM
+STATE_MVOUT_READ    = StateType( 4 ) # MVOUT: Issuing SPM read request
+STATE_MVOUT_RESP    = StateType( 5 ) # MVOUT: Receiving SPM read response and packing
+STATE_MVOUT_WRITE   = StateType( 6 ) # MVOUT: Issuing DRAM write request
+STATE_MVOUT_WAIT    = StateType( 7 ) # MVOUT: Waiting for DRAM write response
+STATE_DONE          = StateType( 8 ) # Signaling command completion


Change from STATE_xxx to STATE_DMA_xxx.

tancheng · 2026-06-15T05:26:49Z

+
+  return mk_bitstruct(new_name, {
+    'dram_data': DramDataType,
+    'dram_mask': DramMaskType,


explain what is dram_mask with comment?

tancheng · 2026-06-15T05:27:00Z

+    'dram_data': DramDataType,
+    'dram_mask': DramMaskType,
+    'spm_data': SpmDataType,
+    'spm_mask': SpmMaskType,


explain what is spm_mask with comment?

tancheng · 2026-06-15T05:35:15Z

Hi @tancheng @BenkangPeng , I summarized two direction of DMA design as below:

Rely on data controller

DMA is added as a new client of the DataMemControllerRTL, where data in the DMA engine communicates directly with the DataMemControllerRTL, and the logic for multiplexing SPM ports is also implemented in that module.
To initiate DMA, the CPU can send dma_mvin or dma_mvout to the CGRA, after which the controller activates the DMA engine by sending start signals.

Pros: Keeps the controller clean; provides a faster path because data does not go through the controller.

Cons: Additional logic is required to feed DMA results into the control memory.

All in controller

All decoding logic is handled in the controller. The logic for handling the logic port should still reside in the DataMemControllerRTL, since the data memory should have its own port multiplexing logic.
The logic of packeting should also be implemented in the controller module.

Pros: Unifies control and data memory within the controller (the controller is already connected to both control and data memory).

Cons: Introduces complex control logic in the controller; results in a slower path.

I prefer the second method but I think there are still some logic should be written in DataMemControllerRTL. WDTY?

We should also include the figure 2 into our

BenkangPeng requested review from HobbitQia and tancheng June 2, 2026 13:55

tancheng reviewed Jun 2, 2026

View reviewed changes

BenkangPeng force-pushed the dma-cgra branch from f41e7a6 to 86f25a4 Compare June 3, 2026 10:29

BenkangPeng commented Jun 3, 2026

View reviewed changes

Comment thread mem/data/DataMemControllerRTL.py Outdated

tancheng reviewed Jun 3, 2026

View reviewed changes

BenkangPeng mentioned this pull request Jun 4, 2026

[CleanUp][NFC] Standardize line endings to LF #294

Merged

BenkangPeng added 13 commits June 4, 2026 17:53

Add the DmaEngine implementation and the test.

308a213

[Test] Update the test of DmaEngine.

90d9eef

Add DMA support to DataMemControllerRTL and implement corresponding t…

5e615e1

…ests.

Add the dma ports into CgraTemplateRTL

30bdc36

Wrap the Cgra and Dma into one single module.

e6c0b3b

[Script] Add the local_CI script file

c3f3dc4

Update .gitignore to ignore the log file

046c860

[Test] Add the test for CgraDmaRTL

4359f1f

[Fix] Fix the bit mismatch error between dma_idx and num_xbar_in_ports.

aff3a8a

[Doc] Add some comments

b2e41e8

[Fix] Fix the bit mismatch by type convertion

5fc388c

Move some constant into common header file

70ae3da

[Refactor] Wrap the signals between dma and dram with SendIfcRTL and …

fc589c5

…RecvIfcRTL. Replace `mem` with `dram` for clarity.

BenkangPeng force-pushed the dma-cgra branch from 86f25a4 to fc589c5 Compare June 4, 2026 10:15

tancheng reviewed Jun 5, 2026

View reviewed changes

Comment thread cgra/CgraDmaRTL.py Outdated

BenkangPeng added 2 commits June 13, 2026 20:56

[Refactor] Update DMA command handling in CgraDmaRTL and CgraTemplate…

17d282c

…RTL. The DMA is connected to the data memory controller indirectly via the controller, with the decoding logic integrated into the controller.

[Refactor] Simplify DMA interface connections in CgraDmaRTL, CgraTemp…

4b98641

…lateRTL, and ControllerRTL.

tancheng reviewed Jun 13, 2026

View reviewed changes

Comment thread cgra/CgraTemplateRTL.py Outdated

Comment thread cgra/CgraTemplateRTL.py Outdated

Comment thread controller/ControllerRTL.py Outdated

Comment thread controller/ControllerRTL.py Outdated

BenkangPeng added 7 commits June 14, 2026 11:10

[Fix] Use Outport instead of Wire in DmaWireIfcRTL to avoid the RTLIR…

e69c3de

… error of pytml verilog backend.

[CleanUp] Remove the unnecessary ports.

82ac18d

[Feature] Introduce DMA data structure and DMA-to-DRAM write request …

1a1172b

…interface for enhanced data transfer capabilities.

[Refactor] Pass DmaCmdType and DmaDataType into DataMemController and…

94cc7d0

… then drives types from them

[Refactor] Update DmaEngineRTL to use DmaDramWrReqIfcRTL for DRAM wri…

65fb185

…te requests and adjust related signal handling for clarity and consistency.

[Refactor] Enhance DMA integration in CgraTemplateRTL and ControllerR…

d2d7e7d

…TL by passing DmaDataType and DmaCmdType as parameters, and updating related type definitions for improved clarity and consistency.

[Refactor] Update CgraDmaRTL to utilize DmaDramWrReqIfcRTL for DRAM w…

28b263b

…rite requests, enhancing type definitions for DmaCmdType and DmaDataType

tancheng reviewed Jun 15, 2026

View reviewed changes

BenkangPeng added 2 commits June 15, 2026 10:19

[Fix] Fix the bitwidth mismatch error between DataType and DmaSpmData…

3af7d8b

…Type

[CleanUp] Update DMA attribute references to use new constants for im…

326167d

…proved clarity

tancheng reviewed Jun 15, 2026

View reviewed changes

	s.recv_from_cpu_pkt //= s.recv_from_cpu_pkt_queue.recv
	s.send_to_cpu_pkt //= s.send_to_cpu_pkt_queue.send


		# Abstract external dram memory interfaces for the internal DMA engine.

		s.dram_rd_req = SendIfcRTL(DmaDramAddrType)

Conversation

BenkangPeng commented Jun 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HobbitQia commented Jun 4, 2026

Uh oh!

tancheng commented Jun 5, 2026

Uh oh!

Uh oh!

HobbitQia commented Jun 7, 2026

Uh oh!

tancheng commented Jun 7, 2026

Uh oh!

HobbitQia commented Jun 8, 2026

Uh oh!

tancheng commented Jun 8, 2026

Uh oh!

HobbitQia commented Jun 8, 2026

Uh oh!

tancheng commented Jun 8, 2026

Uh oh!

HobbitQia commented Jun 8, 2026 • edited by tancheng Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tancheng commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HobbitQia commented Jun 9, 2026

Uh oh!

tancheng commented Jun 9, 2026

Uh oh!

tancheng commented Jun 10, 2026

Uh oh!

BenkangPeng commented Jun 10, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

HobbitQia commented Jun 8, 2026 •

edited by tancheng

Loading

tancheng commented Jun 8, 2026 •

edited

Loading