You have to realize the waters are going to be shared between paddlers and camping groups, Rex groups. If you want some isolated stretch for only fishing and in limited amounts then you should move to the backcountry somewhere.

In my experience fisherman are just as guilty as paddlers at muck up the stream.