My Search Engine - First pre-release

Why?

First of all, why build another metasearch engine? Well, I don’t like the look of searchXNG, and all instances are configured differently. I randomly picked two instances, and one of them had the English language hard-set, while the other one was set to any language. Even this small difference can lead to confusion, as one instance has the results and page only in English, while the other has results in all languages. In a nutshell, every instance is configured differently, settings are unintuitive for non-technical users, and one instance simply cannot handle too many users, which results in users needing to switch to a different instance or host their own, which is inconvenient.

So I went to search for alternatives, and the closest thing to what I wanted was Araa-search. It looks nice and has consistent user settings across different instances, unlike searchXNG. However, it also has the issue of stopping under heavy load, and when I tried to run it on my VPS, it always spiked the CPU to 100% on every search and required 1-2GB of RAM just to remain idle. That’s why I decided to create Warp.

Warp_Search_Mainpage

About Warp

QGato is a metasearch/proxy engine that aims to be user-friendly, themable, scalable, and more resource-efficient than the competition. Here’s a rundown of what I’ve got so far:

Completed Features

Search Capabilities

QGato now supports a wide range of search capabilities:

  • Text Search: Integrated with popular engines like Google, Brave, DuckDuckGo, and LibreX/Y, providing robust text search results. Text Search Example

  • Image Search: Powered by Qwant and Imgur, Warp allows you to find and view images with ease. The integrated image proxy ensures faster loading times and enhances privacy. Image Search Example

  • Video Search: Utilizing the Piped API, video search is seamless, offering quick access to video content without the clutter. Video Search Example

  • Map Results: Map functionality is now available, using OpenStreetMap. This might be changed in the future since it has limited functionality, namely: missing points of interests (restaurants etc.) and no directions. Map Results Example

  • Forums: Currently, this only includes the ability to search Reddit. Other forum results may be added in the future. It’s also using the Reddit API, which isn’t great, so a different approach will likely be implemented. Forums Results Example

  • Files Download: Integrated torrent search allows for easy file downloads directly through torrent sites. As everything else it’s not perfect since it’s not done, but it works. Torrent Results Example

  • Results Caching: Implemented caching for search results, which significantly improves response times and reduces server load. Caching currently saves results for 6 hours. I don’t really like this fixed period of time, so a more dynamic solution will be implemented in the future.

Site Development

To ensure accessibility and performance, I’ve focused on creating a site that works efficiently even without JavaScript:

  • HTML+CSS Site (No JavaScript): The current version of the site works with just HTML and CSS, ensuring it’s lightweight and fast.
  • Proxy-based Image Viewing: To improve anonymous searches images are fetched through a proxy and displayed with direct links to the source pages.

Pending Features

While the current progress is great, there’s still much more planned:

  • JavaScript-Enabled Website: The next step is to introduce a JavaScript version of the site, which will include more interactive features and a smoother user experience.
  • JS Applets for Results: Plans are underway to add JS applets, such as a calculator, that will provide dynamic, in-page functionality.
  • Dynamic Results Loading: I’m working on implementing infinite scroll, where results load dynamically as the user scrolls down the page.
  • Deployment Containers: LXC and Docker containers will be made available to simplify deployment and scaling (I dont like Docker).
  • Automatic Updates: A system for automatic updates is in the works, making it easier to keep QGato up to date.
  • Scalability: Finally, I’m focusing on making the search engine more scalable with crawlers and web servers to handle increased traffic.

Downloads

Update Log Summary

Features Summary

Search Capabilities:

  • Text search using Google, Brave, DuckDuckGo, and LibreX/Y.
  • Image search via Qwant and Imgur.
  • Video search powered by the Piped API.
  • Map results using OpenStreetMap.
  • Forums results using reddit API.
  • Files results (torrent) using torrentgalaxy and thepiratebay.

Site Development:

  • HTML+CSS site available (no JavaScript version).
  • Results caching for faster response times.
  • Proxy-based image viewing with direct links to image source pages.

Additional Features:

  • Files download using torrent sites.
  • Basic configuration setup and initial deployment functionality.

Planned Features

  • JavaScript-enabled website version.
  • JS applets for interactive results (e.g., calculator).
  • Dynamic results loading on scroll.
  • LXC and Docker containers for deployment.
  • Automatic updates.
  • Scalable crawlers and web servers with load balancing.

PS: I’m still thinking about how to name this project. For now, I’m considering “QGato” or “Warp”, so that’s why I sometimes refer to it as “Warp” and sometimes as “QGato”.