You have every right to tell your manager you'd prefer to have the meeting in the office. No telecommuter should be obligated to have co-workers come to the home for meetings or any other purpose - and it doesn't make any difference which sex the telecommuter and the co-workers are. Simply tell your manager you'd rather have the meeting in the office; perhaps you can find a quiet conference room or empty office so you can work without lots of distractions. There's no need to get into any of the history between you and this co-worker. As with all else in telecommuting, the emphasis should be on the work performance.