The Intel® Arria® 10 or the Intel® Cyclone® 10 PCIe* Avalon-MM® DMA version of the IP core does not allow the users to enable the "RX completion buffer overflow" option. This is because the DMA engine ensures that it will not request completions that exceed the available buffer size.
Moreover, per the PCI-SIG specification, the endpoint needs to advertise infinite completion buffer space to the root port.
The "RX completion buffer overflow" option is not allowed to be enabled for The Intel® Arria® 10 or the Intel® Cyclone® 10 PCIe* Avalon-MM® DMA version of the IP core, and it should be left unchecked.