Mouse Drag

Description

Performs a drag operation from one point on the screen to another using exact coordinates. This is useful for drag-and-drop operations, such as moving files, rearranging items, or interacting with sliders and similar UI elements.

The operation consists of:

1. Moving the cursor to the start position
2. Pressing and holding the left mouse button
3. Moving the cursor to the end position
4. Releasing the left mouse button

This endpoint requires specific pixel coordinates for both the start and end positions of the drag operation.

Request Format

{
    "startX": number,  // X coordinate of the starting point
    "startY": number,  // Y coordinate of the starting point
    "endX": number,    // X coordinate of the ending point
    "endY": number     // Y coordinate of the ending point
}
            

Parameters

Example Request

{
  "startX": 100,
  "startY": 200,
  "endX": 300,
  "endY": 400
}

Response Format

{
    "Success": boolean,    // Whether the operation was successful
    "Message": "string",   // Result message or error description
    "Timestamp": "string"  // ISO timestamp of the operation
}
            

Response Fields

Example Success Response

{
  "Success": true,
  "Message": "Mouse drag executed from (100, 200) to (300, 400)",
  "Timestamp": "2023-11-15T14:23:45.123Z"
}

Example Error Response

{
  "Success": false,
  "Message": "Error executing mouse drag: Invalid screen coordinates",
  "Timestamp": "2023-11-15T14:23:45.123Z"
}

Code Examples

import requests
import json

# Define the API endpoint
url = "http://localhost:54321/tools-api/mouse/drag"

# Define the headers
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer your-api-key-here"
}

# Define the request payload
payload = {
    "startX": 100,
    "startY": 200,
    "endX": 300,
    "endY": 400
}

# Send the request
response = requests.post(url, headers=headers, data=json.dumps(payload))

# Parse the response
result = response.json()
print(result)

// Using fetch API
async function performMouseDrag() {
  const url = 'http://localhost:54321/tools-api/mouse/drag';
  
  const payload = {
    startX: 100,
    startY: 200,
    endX: 300,
    endY: 400
  };
  
  const response = await fetch(url, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': 'Bearer your-api-key-here'
    },
    body: JSON.stringify(payload)
  });
  
  const result = await response.json();
  console.log(result);
  return result;
}

using System;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;

public class MouseDragRequest
{
    public int StartX { get; set; }
    public int StartY { get; set; }
    public int EndX { get; set; }
    public int EndY { get; set; }
}

public class ActionResponse
{
    public bool Success { get; set; }
    public string Message { get; set; }
    public DateTime Timestamp { get; set; }
}

public async Task PerformMouseDragAsync()
{
    using (var httpClient = new HttpClient())
    {
        // Set the base address and add authorization header
        httpClient.BaseAddress = new Uri("http://localhost:54321/");
        httpClient.DefaultRequestHeaders.Add("Authorization", "Bearer your-api-key-here");

        // Create the request payload
        var mouseDragRequest = new MouseDragRequest
        {
            StartX = 100,
            StartY = 200,
            EndX = 300,
            EndY = 400
        };

        // Serialize the request
        var content = new StringContent(
            JsonSerializer.Serialize(mouseDragRequest),
            Encoding.UTF8,
            "application/json");

        // Send the request
        var response = await httpClient.PostAsync("tools-api/mouse/drag", content);
        
        // Process the response
        if (response.IsSuccessStatusCode)
        {
            var responseContent = await response.Content.ReadAsStringAsync();
            return JsonSerializer.Deserialize(responseContent, 
                new JsonSerializerOptions { PropertyNameCaseInsensitive = true });
        }
        else
        {
            throw new Exception($"Error: {response.StatusCode}");
        }
    }
}

Examples & Use Cases

Example 1: Dragging a File to a Folder

You can use this endpoint to drag a file icon to a folder location:

{
  "startX": 150,  // X coordinate of the file icon
  "startY": 200,  // Y coordinate of the file icon
  "endX": 400,    // X coordinate of the target folder
  "endY": 300     // Y coordinate of the target folder
}

Example 2: Using a Slider Control

This endpoint can be used to operate slider controls in applications:

{
  "startX": 200,  // X coordinate of the slider current position
  "startY": 250,  // Y coordinate of the slider
  "endX": 350,    // X coordinate of the desired slider position
  "endY": 250     // Y coordinate of the slider (usually the same as startY)
}

Example 3: Rearranging Items in a List

You can rearrange items in a list by dragging them to a new position:

{
  "startX": 300,  // X coordinate of the item to move
  "startY": 150,  // Y coordinate of the item to move
  "endX": 300,    // X coordinate of the target position (often the same)
  "endY": 250     // Y coordinate of the target position
}

Notes

• The coordinates are measured in pixels from the top-left corner of the screen (0,0).
• The drag operation is performed with the left mouse button.
• For more complex drag operations where you don't know the exact coordinates, consider using the AI vision-based endpoint /tools-api/mouse/drag-by-description instead.
• The operation includes a small delay (100ms) between pressing the button and moving to the end position to ensure stability.