Well, I had similar issue, so I copied admin-ajax.php to my theme directory and removed the unneeded stuff. Then use it for your ajax calls instead of the wp-admin one.
Reffer to the file:
<?php get_template_directory_uri(); ?>/admin-ajax.php
This is my admin-ajax.php according the latest WP 3.4.2:
<?php
/**
* WordPress AJAX Process Execution.
*/
define( 'DOING_AJAX', true );
/** Load WordPress Bootstrap */
require_once( '../../../wp-load.php' );
/** Allow for cross-domain requests (from the frontend). */
send_origin_headers();
// Require an action parameter
if ( empty( $_REQUEST['action'] ) )
die( '0' );
@header( 'Content-Type: text/html; charset=' . get_option( 'blog_charset' ) );
@header( 'X-Robots-Tag: noindex' );
send_nosniff_header();
add_action( 'wp_ajax_nopriv_autosave', 'wp_ajax_nopriv_autosave', 1 );
if ( is_user_logged_in() )
do_action( 'wp_ajax_' . $_REQUEST['action'] ); // Authenticated actions
else
do_action( 'wp_ajax_nopriv_' . $_REQUEST['action'] ); // Non-admin actions
// Default status
die( '0' );
Hope this helps!