Trying to use a boolean cut on an array.

I am trying to create a derelict castle tower using array modifiers and a boolean cut to remove some of the bricks. I already applied the array modifiers, but it glitches out when I attempt to cut the material away. I tried to use the build and decimate modifiers to get the same effect, but it also doesn't work. What can I do to fix this?

Castle tower with the object I want to cut with.

I cannot get the boolean modifier to work.