The fan design itself can contribute significantly to the noise.
I did some measurements with a 6 inch fan recently & found a lot of the noise can be cut by using an insulated duct. This works well because significant proportion of the noise propagates axially or up and down the duct.
AC Infinity T6 evaluation and test data.
The insulated duct is basically just the regular duct with about 1.5" of pink insulation and a plastic skin over that.
Hope this gives you some useful ideas
Cheers
G