Performance issues using SQL CLR - High PAGEIOLATCH_SH

Dear community, 

let me describe my problem: 

Query: 

SELECT t.value1, t.value2, x.valueA

from My300GBTable t

CROSS APPLY MyTableValuedFunction(varbinary field from table t) x

Usually, this works fine. But in some cases the plan generation takes >30 minutes. The system is showing 0% CPU, 3mb/s disk read and low memory pressure (~700GB available). IO System is 72 disk array. Disk queue length on logical disk is around 1.

dm_exec_session_wait_stats shows high PAGEIOLATCH_SH for the session. 

Once the plan is ready, queries are running fast.

Any ideas how to troubleshoot that?