Visitor numbers at this year's London International Wine Fair (LIWF) fell year-on-year as organisers branded it "the most challenging show to date".