cuqmbr/rss-bridge
1
0
Fork 0
mirror of https://github.com/RSS-Bridge/rss-bridge.git synced 2025-04-23 15:06:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' of github.com:RSS-Bridge/rss-bridge into testing

Browse Source
This commit is contained in:
Teromene 2016-08-30 10:46:23 +02:00
commit 10fe286a4a
175 changed files with 6851 additions and 8711 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.gitignore vendored
View File

@ -228,3 +228,8 @@ pip-log.txt
/whitelist.txt
DEBUG
phpunit*.phar
######################
## VisualStudioCode ##
######################
.vscode/*

15
.travis.yml Normal file
View File

@ -0,0 +1,15 @@
language: php
php:
- '5.4'
- '5.5'
- '5.6'
- '7.0'
- hhvm
- nightly
install:
- pear install PHP_CodeSniffer
script:
- phpenv rehash
- phpcs . --standard=phpcs.xml --warning-severity=0 --extensions=php -p

186
CHANGELOG.md
View File

@ -3,27 +3,169 @@ rss-bridge Changelog
Alpha 0.1
===
* Firt tagged version.
* Includes refactoring.
* Unstable.
* First tagged version.
* Includes refactoring.
* Unstable.
Current development version
Alpha 0.2 (current development version)
===
* Corrected GoogleBridge (URI extraction was incorrect)
* Corrected ATOM format:
* mime-type was incorrect
* Hyperlinks were not clickable.
* non-UTF8 characters are now properly filtered.
* Corrected HTML format output:
* Hyperlinks were not clickable.
* Corrected error message when SimpleHtmlDom library is not installed.
* Added changelog.
* (many many fixes on bridges)
* Media RSS format
* input types (list)
* cache purge function
* refactoring
Alpha 0.2
===
-TBA-
## Important changes
* RSS-Bridge has been [UNLICENSED](UNLICENSE)
* RSS-Bridge is now a community-managed project on [GitHub](https://github.com/rss-bridge/rss-bridge)
* RSS-Bridge now has a [Wiki](https://github.com/rss-bridge/rss-bridge/wiki)
## General changes
* Added [CHANGELOG](CHANGELOG.md) (this file)
* Added [PHP Simple HTML DOM Parser](http://simplehtmldom.sourceforge.net) to [vendor](vendor/simplehtmldom/)
* Added cache purging function (cache will be force-purged after 24 hours or as defined by bridge)
* Added new format [MrssFormat](formats/MrssFormat.php)
* Added parameter `author` - for display of the feed author name - to all formats
* Added new abstractions of the BridgeInterface:
- [HttpCachingBridgeAbstract](https://github.com/RSS-Bridge/rss-bridge/wiki/Bridge-API)
- [RssExpander](https://github.com/RSS-Bridge/rss-bridge/wiki/Bridge-API)
* Added optional support for proxy usage on each individual bridge
* Added support for [custom bridge parameter](https://github.com/RSS-Bridge/rss-bridge/wiki/BridgeAbstract#format-specifications) (text, number, list, checkbox)
* Changed design of the welcome screen
* Changed design of HtmlFormat
* Changed behavior of debug mode:
- Enable debug mode by placing a file called "DEBUG" in the root folder
- Debug mode automatically disables caching
* Changed implementation of bridges - see [Wiki](https://github.com/rss-bridge/rss-bridge/wiki)
- Changed comment-style metadata to public function [`loadMetadatas`](https://github.com/RSS-Bridge/rss-bridge/wiki/BridgeAbstract#the-loadmetadatas-function)
- Added support for multiple utilizations per bridge
* Many bug fixes
## Modified Bridges
* FlickrExploreBridge
* GoogleSearchBridge
* TwitterBridge
## New Bridges
* ABCTabsBridge
* AcrimedBridge
* AllocineFRBridge
* AnimeUltimeBridge
* ArstechnicaBridge
* Arte7Bridge
* AskfmBridge
* BandcampBridge
* BastaBridge
* BlaguesDeMerdeBridge
* BooruprojectBridge
* CADBridge
* CNETBridge
* CastorusBridge
* CollegeDeFranceBridge
* CommonDreamsBridge
* CopieDoubleBridge
* CourrierInternationalBridge
* CpasbienBridge
* CryptomeBridge
* DailymotionBridge
* DanbooruBridge
* DansTonChatBridge
* DauphineLibereBridge
* DemoBridge
* DeveloppezDotComBridge
* DilbertBridge
* DollbooruBridge
* DuckDuckGoBridge
* EZTVBridge
* EliteDangerousGalnetBridge
* ElsevierBridge
* EstCeQuonMetEnProdBridge
* FacebookBridge
* FierPandaBridge
* FlickrTagBridge
* FootitoBridge
* FourchanBridge
* Freenews
* FuturaSciencesBridge
* GBAtempBridge
* Gawker
* GelbooruBridge
* GiphyBridge
* GithubIssueBridge
* GitlabCommitsBridge
* GizmodoFRBridge
* GooglePlusPostBridge
* GuruMedBridge
* HDWallpapersBridge
* HentaiHavenBridge
* IdenticaBridge
* InstagramBridge
* IsoHuntBridge
* JapanExpoBridge
* KonachanBridge
* KoreusBridge
* KununuBridge
* LWNprevBridge
* LeBonCoinBridge
* LeJournalDuGeekBridge
* LeMondeInformatiqueBridge
* Les400Culs
* LesJoiesDuCodeBridge
* LichessBridge
* LinkedInCompany
* LolibooruBridge
* MangareaderBridge
* MilbooruBridge
* MondeDiploBridge
* MsnMondeBridge
* MspabooruBridge
* NakedSecurityBridge
* NasaApodBridge
* NeuviemeArtBridge
* NextInpactBridge
* NextgovBridge
* NiceMatinBridge
* NovelUpdatesBridge
* NumeramaBridge
* OpenClassroomsBridge
* ParuVenduImmoBridge
* PickyWallpapersBridge
* PinterestBridge
* PlanetLibreBridge
* ProjectMGameBridge
* RTBFBridge
* Releases3DSBridge
* ReporterreBridge
* Rue89Bridge
* Rule34Bridge
* Rule34pahealBridge
* SafebooruBridge
* SakugabooruBridge
* ScmbBridge
* ScoopItBridge
* SensCritiqueBridge
* Sexactu
* ShanaprojectBridge
* SiliconBridge
* SoundcloudBridge
* StripeAPIChangeLogBridge
* SuperbWallpapersBridge
* T411Bridge
* TagBoardBridge
* TbibBridge
* TheCodingLoveBridge
* TheHackerNewsBridge
* TheOatMealBridge
* ThePirateBayBridge
* TwitchApiBridge
* UnsplashBridge
* ViadeoCompany
* VineBridge
* VkBridge
* WallpaperStopBridge
* WeLiveSecurityBridge
* WhydBridge
* WikipediaBridge
* WordPressBridge
* WorldOfTanks
* XbooruBridge
* YandereBridge
* YoutubeBridge
* ZDNetBridge
* ZatazBridge
* ZoneTelechargementBridge

105
CREATE_BRIDGE.md
View File

@ -1,105 +0,0 @@
# Howto create a bridge
A bridge is an interface that allows rss-bridge to create a RSS feed from a website.
The bridge is a PHP file, located in the `bridges/` folder.
##Specifications
A rss bridge must extend the `BridgeAbstract` class, and implement the following functions :
* The `loadMetadatas` function, described below,
* The `getCacheDuration` function, describing the time during which rss-bridge will output cached values instead of re-generating a RSS feed.
* The `collectData` function, also described below.
##The `collectData` function
This function takes as a parameter an array called `$param`, that is automatically filled with values from the user, according to the values setted in `loadMetadatas`.
This function is the place where all the website scrapping and the RSS feed generation process will go.
The RSS elements are stored in the class variable `items[]`.
Every RSS element is an instance of the `Item` class.
##The `loadMetadatas` function
This function is the one used by rss-bridge core to determine the name, maintainer name, website, last updated date... of the bridge, and the user parameters.
### Basic metadatas.
The basic metadatas are the following :
```PHP
$this->maintainer
$this->name
$this->uri
$this->description
$this->update
```
The default values are the following :
```PHP
$this->name = "Unnamed bridge";
$this->uri = "";
$this->description = 'No description provided';
$this->maintainer = 'No maintainer';
```
### Parameters
Parameters use a JSON-like format, which is parsed and transformed to HTML `<form>` by rss-bridge.
These datas goes into the `$this->parameters` array, which is not mandatory if your bridge doesn't take any parameter.
Every possible usage of a bridge is an array element.
The array can be a key-based array, but it is not necessary. The following syntaxes are hereby correct :
```PHP
$this->parameters[] = ...
$this->parameters['First usage of my bridge'] = ...
```
It is worth mentionning that you can also define a set of parameters that will be applied to every possible utilisation of your bridge.
To do this, just create a parameter array with the `global` key.
### Format specifications
Every `$this->parameters` element is a JSON array (`[ ... ]`) containing every input.
It needs the following elements to be setted :
* name, the input name as displayed to the user
* identifier, which will be the key in the `$param` array for the corresponding data.
Hence, the most basic parameter definition is the following:
```PHP
$this->parameters =
'[
{
"name" : "Username",
"identifier" : "u"
}
]';
```
####Optional parameters
Here is a list of optional parameters for the input :
Parameter Name | Parameter values | Description
---------------|------------------|------------
type|text, number, list, checkbox| Type of the input, default is text
required| true | Set this if you want your attribute to be required
values| [ {"name" : option1Name, "value" : "option1Value"}, ...] | Values list, required with the 'list' type
title| text | Will be shown as tooltip when mouse-hovering over the input
#### Guidelines
* scripts (eg. Javascript) must be stripped out. Make good use of `strip_tags()` and `preg_replace()`
* bridge must present data within 8 seconds (adjust iterators accordingly)
* cache timeout must be fine-tuned so that each refresh can provide 1 or 2 new elements on busy periods
* `<audio>` and `<video>` must not autoplay. Seriously.
* do everything you can to extract valid timestamps. Translate formats, use API, exploit sitemap, whatever. Free the data!
* don't create duplicates. If the website runs on WordPress, use the generic WordPress bridge if possible.
* maintain efficient and well-commented code :wink:

35
README.md
View File

@ -1,5 +1,6 @@
rss-bridge
===
[![LICENSE](https://img.shields.io/badge/license-UNLICENSE-blue.svg)](UNLICENSE)
rss-bridge is a PHP project capable of generating ATOM feeds for websites which don't have one.
@ -19,9 +20,7 @@ Supported sites/pages (main)
* `OpenClassrooms`: Lastest tutorials from [fr.openclassrooms.com](http://fr.openclassrooms.com/)
* `Pinterest`: Most recent photos from user or search
* `ScmbBridge`: Newest stories from [secouchermoinsbete.fr](http://secouchermoinsbete.fr/)
* `WikipediaENLatest`: highlighted articles from Wikipedia in English
* `WikipediaFRLatest`: highlighted articles from Wikipedia in French
* `WikipediaEOLatest`: highlighted articles from Wikipedia in Esperanto
* `Wikipedia`: highlighted articles from [Wikipedia](https://wikipedia.org/) in English, German, French or Esperanto
* `Bandcamp` : Returns last release from [bandcamp](https://bandcamp.com/) for a tag
* `ThePirateBay` : Returns the newest indexed torrents from [The Pirate Bay](https://thepiratebay.se/) with keywords
* `Facebook` : Returns the latest posts on a page or profile on [Facebook](https://facebook.com/)
@ -33,6 +32,7 @@ Output format
Output format can take several forms:
* `Atom` : ATOM Feed, for use in RSS/Feed readers
* `Mrss` : MRSS Feed, for use in RSS/Feed readers
* `Json` : Json, for consumption by other applications.
* `Html` : Simple html page.
* `Plaintext` : raw text (php object, as returned by print_r)
@ -42,11 +42,11 @@ Screenshot
Welcome screen:
![Screenshot](http://sebsauvage.net/galerie/photos/Bordel/rss-bridge-screenshot-3.png)
![Screenshot](https://github.com/RSS-Bridge/rss-bridge/wiki/images/screenshot_rss-bridge_welcome.png)
Minecraft hashtag (#Minecraft) search on Twitter, in ATOM format (as displayed by Firefox):
RSS-Bridge hashtag (#rss-bridge) search on Twitter, in ATOM format (as displayed by Firefox):
![Screenshot](http://sebsauvage.net/galerie/photos/Bordel/rss-bridge-screenshot-2-twitter-hashtag.png)
![Screenshot](https://github.com/RSS-Bridge/rss-bridge/wiki/images/screenshot_twitterbridge_atom.png)
Requirements
===
@ -60,16 +60,21 @@ Enabling/Disabling bridges
By default, the script creates `whitelist.txt` and adds the main bridges (see above). `whitelist.txt` is ignored by git, you can edit it:
* to enable extra bridges (one bridge per line)
* to disable main bridges (remove the line)
* to enable all bridges (just one wildcard `*` as file content)
New bridges are disabled by default, so make sure to check regularly what's new and whitelist what you want !
Author
New bridges are disabled by default, so make sure to check regularly what's new and whitelist what you want!
Deploy
===
I'm sebsauvage, webmaster of [sebsauvage.net](http://sebsauvage.net), author of [Shaarli](http://sebsauvage.net/wiki/doku.php?id=php:shaarli) and [ZeroBin](http://sebsauvage.net/wiki/doku.php?id=php:zerobin).
[![Deploy on Scalingo](https://cdn.scalingo.com/deploy/button.svg)](https://my.scalingo.com/deploy?source=https://github.com/sebsauvage/rss-bridge)
Authors
===
We are RSS Bridge Community, a group of developers continuing the project initiated by sebsauvage, webmaster of [sebsauvage.net](http://sebsauvage.net), author of [Shaarli](http://sebsauvage.net/wiki/doku.php?id=php:shaarli) and [ZeroBin](http://sebsauvage.net/wiki/doku.php?id=php:zerobin).
Patch/contributors :
* Yves ASTIER ([Draeli](https://github.com/Draeli)) : PHP optimizations, fixes, dynamic brigde/format list with all stuff behind and extend cache system. Mail : contact@yves-astier.com
* Yves ASTIER ([Draeli](https://github.com/Draeli)) : PHP optimizations, fixes, dynamic brigde/format list with all stuff behind and extend cache system. Mail : contact /at\ yves-astier.com
* [Mitsukarenai](https://github.com/Mitsukarenai) : Initial inspiration, collaborator
* [ArthurHoaro](https://github.com/ArthurHoaro)
* [BoboTiG](https://github.com/BoboTiG)
@ -101,8 +106,9 @@ Patch/contributors :
* [gsurrel](https://github.com/gsurrel)
* [kraoc](https://github.com/kraoc)
* [lagaisse](https://github.com/lagaisse)
* [az5he6ch](https://github.com/az5he6ch)
License
Licenses
===
Code is [Public Domain](UNLICENSE).
@ -111,9 +117,10 @@ Including `PHP Simple HTML DOM Parser` under the [MIT License](http://opensource
Technical notes
===
* There is a cache so that source services won't ban you even if you hammer the rss-bridge with requests. Each bridge has a different duration for the cache. The `cache` subdirectory will be automatically created and cached objects older than 24 hours get purged.
* To implement a new rss-bridge, [follow the specifications](CREATE_BRIDGE.md) and take a look at existing bridges for examples.
* There is a cache so that source services won't ban you even if you hammer the rss-bridge with requests. Each bridge can have a different duration for the cache. The `cache` subdirectory will be automatically created and cached objects older than 24 hours get purged.
* To implement a new Bridge, [follow the specifications](https://github.com/RSS-Bridge/rss-bridge/wiki/Bridge-API) and take a look at existing Bridges for examples.
* To enable debug mode (disabling cache and enabling error reporting), create an empty file named `DEBUG` in the root directory (next to `index.php`).
* For more information refer to the [Wiki](https://github.com/RSS-Bridge/rss-bridge/wiki)
Rant
===

49
bridges/ABCTabsBridge.php
View File

@ -1,48 +1,25 @@
<?php
class ABCTabsBridge extends BridgeAbstract{
private $request;
public $maintainer = "kranack";
public $name = "ABC Tabs Bridge";
public $uri = "http://www.abc-tabs.com/";
public $description = "Returns 22 newest tabs";
public function loadMetadatas() {
$this->maintainer = "kranack";
$this->name = "ABC Tabs Bridge";
$this->uri = "http://www.abc-tabs.com/";
$this->description = "Returns 22 newest tabs";
$this->update = "2014-07-23";
}
public function collectData(array $param){
public function collectData(){
$html = '';
$html = $this->file_get_html('http://www.abc-tabs.com/tablatures/nouveautes.html') or $this->returnError('No results for this query.', 404);
$html = $this->getSimpleHTMLDOM('http://www.abc-tabs.com/tablatures/nouveautes.html') or $this->returnClientError('No results for this query.');
$table = $html->find('table#myTable', 0)->children(1);
foreach ($table->find('tr') as $tab)
{
$item = new \Item();
$item->name = $tab->find('td', 1)->plaintext . ' - ' . $tab->find('td', 2)->plaintext;
$item->title = $tab->find('td', 1)->plaintext . ' - ' . $tab->find('td', 2)->plaintext;
$item->content = 'Le ' . $tab->find('td', 0)->plaintext . '<br> Par: ' . $tab->find('td', 5)->plaintext . '<br> Type: ' . $tab->find('td', 3)->plaintext;
$item->id = 'http://www.abc-tabs.com' . $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$item->uri = 'http://www.abc-tabs.com' . $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$item = array();
$item['author'] = $tab->find('td', 1)->plaintext . ' - ' . $tab->find('td', 2)->plaintext;
$item['title'] = $tab->find('td', 1)->plaintext . ' - ' . $tab->find('td', 2)->plaintext;
$item['content'] = 'Le ' . $tab->find('td', 0)->plaintext . '<br> Par: ' . $tab->find('td', 5)->plaintext . '<br> Type: ' . $tab->find('td', 3)->plaintext;
$item['id'] = 'http://www.abc-tabs.com' . $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$item['uri'] = 'http://www.abc-tabs.com' . $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$this->items[] = $item;
}
}
public function getName() {
return "ABC Tabs Bridge";
}
public function getURI() {
return "http://www.abc-tabs.com/";
}
public function getCacheDuration(){
return 3600; // 1 hour
}
}

44
bridges/AcrimedBridge.php
View File

@ -1,22 +1,18 @@
<?php
class AcrimedBridge extends RssExpander{
public function loadMetadatas() {
$this->maintainer = "qwertygc";
$this->name = "Acrimed Bridge";
$this->uri = "http://www.acrimed.org/";
$this->description = "Returns the newest articles.";
$this->update = "2014-05-25";
public $maintainer = "qwertygc";
public $name = "Acrimed Bridge";
public $uri = "http://www.acrimed.org/";
public $description = "Returns the newest articles.";
public function collectData(){
$this->collectExpandableDatas("http://www.acrimed.org/spip.php?page=backend");
}
public function collectData(array $param){
parent::collectExpandableDatas($param, "http://www.acrimed.org/spip.php?page=backend");
}
protected function parseRSSItem($newsItem) {
$hs = new HTMLSanitizer();
@ -24,34 +20,22 @@ class AcrimedBridge extends RssExpander{
$namespaces = $newsItem->getNameSpaces(true);
$dc = $newsItem->children($namespaces['dc']);
$item = new Item();
$item->uri = trim($newsItem->link);
$item->title = trim($newsItem->title);
$item->timestamp = strtotime($dc->date);
$item = array();
$item['uri'] = trim($newsItem->link);
$item['title'] = trim($newsItem->title);
$item['timestamp'] = strtotime($dc->date);
$articlePage = $this->file_get_html($newsItem->link);
$articlePage = $this->getSimpleHTMLDOM($newsItem->link);
$article = $hs->sanitize($articlePage->find('article.article1', 0)->innertext);
$article = HTMLSanitizer::defaultImageSrcTo($article, "http://www.acrimed.org/");
$item->content = $article;
$item['content'] = $article;
return $item;
}
public function getName() {
return "Acrimed Bridge";
}
public function getURI() {
return "http://www.acrimed.org/";
}
public function getCacheDuration(){
return 4800; // 2 hours
}

97
bridges/AllocineFRBridge.php
View File

@ -1,57 +1,84 @@
<?php
class AllocineFRBridge extends BridgeAbstract{
private $_URL = "http://www.allocine.fr/video/programme-12284/saison-24580/";
private $_NOM = "Faux Raccord";
public function loadMetadatas() {
public $maintainer = "superbaillot.net";
public $name = "Allo Cine Bridge";
public $uri = "http://www.allocine.fr";
public $description = "Bridge for allocine.fr";
public $parameters = array( array(
'category'=>array(
'name'=>'category',
'type'=>'list',
'required'=>true,
'exampleValue'=>'Faux Raccord',
'title'=>'Select your category',
'values'=>array(
'Faux Raccord'=>'faux-raccord',
'Top 5'=>'top-5',
'Tueurs en Séries'=>'tueurs-en-serie'
)
)
));
$this->maintainer = "superbaillot.net";
$this->name = "Allo Cine : Faux Raccord";
$this->uri = "http://www.allocine.fr/video/programme-12284/saison-24580/";
$this->description = "Allo Cine : Faux Raccord";
$this->update = "07/11/2013";
public function getURI(){
switch($this->getInput('category')){
case 'faux-raccord':
$uri = 'http://www.allocine.fr/video/programme-12284/saison-24580/';
break;
case 'top-5':
$uri = 'http://www.allocine.fr/video/programme-12299/saison-22542/';
break;
case 'tueurs-en-serie':
$uri = 'http://www.allocine.fr/video/programme-12286/saison-22938/';
break;
}
}
return $uri;
}
public function getName(){
return $this->name.' : '
.array_search(
$this->getInput('category'),
$this->parameters[$this->queriedContext]['category']['values']
);
}
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->getURI())
or $this->returnServerError("Could not request ".$this->getURI()." !");
$category=array_search(
$this->getInput('category'),
$this->parameters[$this->queriedContext]['category']['values']
);
public function collectData(array $param){
$html = $this->file_get_html($this->_URL) or $this->returnError('Could not request Allo cine.', 404);
foreach($html->find('figure.media-meta-fig') as $element)
{
$item = new Item();
$titre = $element->find('div.titlebar h3.title a', 0);
$item = array();
$title = $element->find('div.titlebar h3.title a', 0);
$content = trim($element->innertext);
$figCaption = strpos($content, $this->_NOM);
$figCaption = strpos($content, $category);
if($figCaption !== false)
{
$content = str_replace('src="/', 'src="http://www.allocine.fr/',$content);
$content = str_replace('href="/', 'href="http://www.allocine.fr/',$content);
$content = str_replace('src=\'/', 'src=\'http://www.allocine.fr/',$content);
$content = str_replace('href=\'/', 'href=\'http://www.allocine.fr/',$content);
$item->content = $content;
$item->title = trim($titre->innertext);
$item->uri = "http://www.allocine.fr" . $titre->href;
$content = str_replace('src="/', 'src="http://www.allocine.fr/', $content);
$content = str_replace('href="/', 'href="http://www.allocine.fr/', $content);
$content = str_replace('src=\'/', 'src=\'http://www.allocine.fr/', $content);
$content = str_replace('href=\'/', 'href=\'http://www.allocine.fr/', $content);
$item['content'] = $content;
$item['title'] = trim($title->innertext);
$item['uri'] = "http://www.allocine.fr" . $title->href;
$this->items[] = $item;
}
}
}
public function getName(){
return 'Allo Cine : ' . $this->_NOM;
}
public function getURI(){
return $this->_URL;
}
public function getCacheDuration(){
return 25200; // 7 hours
}
public function getDescription(){
return "Allo Cine : " . $this->_NOM . " via rss-bridge";
}
}
?>

57
bridges/AllocineT5Bridge.php
View File

@ -1,57 +0,0 @@
<?php
class AllocineT5Bridge extends BridgeAbstract{
private $_URL = "http://www.allocine.fr/video/programme-12299/saison-22542/";
private $_NOM = "Top 5";
public function loadMetadatas() {
$this->maintainer = "superbaillot.net";
$this->name = "Allo Cine : Top 5";
$this->uri = "http://www.allocine.fr/video/programme-12299/saison-22542/";
$this->description = "Allo Cine : Top 5 via rss-bridge";
$this->update = "07/11/2013";
}
public function collectData(array $param){
$html = $this->file_get_html($this->_URL) or $this->returnError('Could not request Allo cine.', 404);
foreach($html->find('figure.media-meta-fig') as $element)
{
$item = new Item();
$titre = $element->find('div.titlebar h3.title a', 0);
$content = trim($element->innertext);
$figCaption = strpos($content, $this->_NOM);
if($figCaption !== false)
{
$content = str_replace('src="/', 'src="http://www.allocine.fr/',$content);
$content = str_replace('href="/', 'href="http://www.allocine.fr/',$content);
$content = str_replace('src=\'/', 'src=\'http://www.allocine.fr/',$content);
$content = str_replace('href=\'/', 'href=\'http://www.allocine.fr/',$content);
$item->content = $content;
$item->title = trim($titre->innertext);
$item->uri = "http://www.allocine.fr" . $titre->href;
$this->items[] = $item;
}
}
}
public function getName(){
return 'Allo Cine : ' . $this->_NOM;
}
public function getURI(){
return $this->_URL;
}
public function getCacheDuration(){
return 25200; // 7 hours
}
public function getDescription(){
return "Allo Cine : " . $this->_NOM . " via rss-bridge";
}
}
?>

57
bridges/AllocineTueursEnSerieBridge.php
View File

@ -1,57 +0,0 @@
<?php
class AllocineTueursEnSerieBridge extends BridgeAbstract{
private $_URL = "http://www.allocine.fr/video/programme-12286/saison-22938/";
private $_NOM = "Tueurs en Séries";
public function loadMetadatas() {
$this->maintainer = "superbaillot.net";
$this->name = "Allo Cine : Tueurs En Serie";
$this->uri = "http://www.allocine.fr/video/programme-12286/saison-22938/";
$this->description = "Allo Cine : Tueurs En Serie";
$this->update = "12/11/2013";
}
public function collectData(array $param){
$html = $this->file_get_html($this->_URL) or $this->returnError('Could not request Allo cine.', 404);
foreach($html->find('figure.media-meta-fig') as $element)
{
$item = new Item();
$titre = $element->find('div.titlebar h3.title a', 0);
$content = trim($element->innertext);
$figCaption = strpos($content, $this->_NOM);
if($figCaption !== false)
{
$content = str_replace('src="/', 'src="http://www.allocine.fr/',$content);
$content = str_replace('href="/', 'href="http://www.allocine.fr/',$content);
$content = str_replace('src=\'/', 'src=\'http://www.allocine.fr/',$content);
$content = str_replace('href=\'/', 'href=\'http://www.allocine.fr/',$content);
$item->content = $content;
$item->title = trim($titre->innertext);
$item->uri = "http://www.allocine.fr" . $titre->href;
$this->items[] = $item;
}
}
}
public function getName(){
return 'Allo Cine : ' . $this->_NOM;
}
public function getURI(){
return $this->_URL;
}
public function getCacheDuration(){
return 25200; // 7 hours
}
public function getDescription(){
return "Allo Cine : " . $this->_NOM . " via rss-bridge";
}
}
?>

128
bridges/AnimeUltimeBridge.php
View File

@ -1,60 +1,32 @@
<?php
class AnimeUltimeBridge extends BridgeAbstract {
public $maintainer = 'ORelio';
public $name = 'Anime-Ultime';
public $uri = 'http://www.anime-ultime.net/';
public $description = 'Returns the 10 newest releases posted on Anime-Ultime';
public $parameters = array( array(
'type'=>array(
'name'=>'Type',
'type'=>'list',
'values'=>array(
'Everything'=>'',
'Anime'=>'A',
'Drama'=>'D',
'Tokusatsu'=>'T'
)
)
));
private $filter = 'Releases';
public function loadMetadatas() {
$this->maintainer = 'ORelio';
$this->name = 'Anime-Ultime';
$this->uri = 'http://www.anime-ultime.net/';
$this->description = 'Returns the 10 newest releases posted on Anime-Ultime';
$this->update = '2015-10-30';
$this->parameters[] =
'[
{
"name" : "Type",
"type" : "list",
"identifier" : "type",
"values" :
[
{
"name" : "Everything",
"value" : ""
},
{
"name" : "Anime",
"value" : "A"
},
{
"name" : "Drama",
"value" : "D"
},
{
"name" : "Tokusatsu",
"value" : "T"
}
]
}
]';
}
public function collectData(array $param) {
public function collectData(){
//Add type filter if provided
$typeFilter = '';
if (!empty($param['type'])) {
if ($param['type'] == 'A' || $param['type'] == 'D' || $param['type'] == 'T') {
$typeFilter = $param['type'];
if ($typeFilter == 'A') { $this->filter = 'Anime'; }
if ($typeFilter == 'D') { $this->filter = 'Drama'; }
if ($typeFilter == 'T') { $this->filter = 'Tokusatsu'; }
} else $this->returnError('The provided type filter is invalid. Expecting A, D, T, or no filter', 400);
}
$typeFilter = array_search(
$this->getInput('type'),
$this->parameters[$this->queriedContext]['type']['values']
);
//Build date and filters for making requests
$thismonth = date('mY').$typeFilter;
@ -65,9 +37,9 @@ class AnimeUltimeBridge extends BridgeAbstract {
foreach (array($thismonth, $lastmonth) as $requestFilter) {
//Retrive page contents
$website = 'http://www.anime-ultime.net/';
$url = $website.'history-0-1/'.$requestFilter;
$html = $this->file_get_html($url) or $this->returnError('Could not request Anime-Ultime: '.$url, 500);
$url = $this->uri.'history-0-1/'.$requestFilter;
$html = $this->getSimpleHTMLDOM($url)
or $this->returnServerError('Could not request Anime-Ultime: '.$url);
//Relases are sorted by day : process each day individually
foreach ($html->find('div.history', 0)->find('h3') as $daySection) {
@ -75,7 +47,9 @@ class AnimeUltimeBridge extends BridgeAbstract {
//Retrieve day and build date information
$dateString = $daySection->plaintext;
$day = intval(substr($dateString, strpos($dateString, ' ') + 1, 2));
$item_date = strtotime(str_pad($day, 2, '0', STR_PAD_LEFT).'-'.substr($requestFilter, 0, 2).'-'.substr($requestFilter, 2, 4));
$item_date = strtotime(str_pad($day, 2, '0', STR_PAD_LEFT).'-'
.substr($requestFilter, 0, 2).'-'
.substr($requestFilter, 2, 4));
$release = $daySection->next_sibling()->next_sibling()->first_child(); //<h3>day</h3><br /><table><tr> <-- useful data in table rows
//Process each release of that day, ignoring first table row: contains table headers
@ -84,9 +58,8 @@ class AnimeUltimeBridge extends BridgeAbstract {
//Retrieve metadata from table columns
$item_link_element = $release->find('td', 0)->find('a', 0);
$item_uri = $website.$item_link_element->href;
$item_uri = $this->uri.$item_link_element->href;
$item_name = html_entity_decode($item_link_element->plaintext);
$item_image = $website.substr($item_link_element->onmouseover, 37, strpos($item_link_element->onmouseover, ' ', 37) - 37);
$item_episode = html_entity_decode(str_pad($release->find('td', 1)->plaintext, 2, '0', STR_PAD_LEFT));
$item_fansub = $release->find('td', 2)->plaintext;
$item_type = $release->find('td', 4)->plaintext;
@ -94,25 +67,35 @@ class AnimeUltimeBridge extends BridgeAbstract {
if (!empty($item_uri)) {
//Retrieve description from description page and convert relative image src info absolute image src
$html_item = file_get_contents($item_uri) or $this->returnError('Could not request Anime-Ultime: '.$item_uri, 500);
$item_description = substr($html_item, strpos($html_item, 'class="principal_contain" align="center">') + 41);
$item_description = substr($item_description, 0, strpos($item_description, '<div id="table">'));
$item_description = str_replace('src="images', 'src="'.$website.'images', $item_description);
$html_item = $this->getContents($item_uri)
or $this->returnServerError('Could not request Anime-Ultime: '.$item_uri);
$item_description = substr(
$html_item,
strpos($html_item, 'class="principal_contain" align="center">')
+ 41
);
$item_description = substr($item_description,
0,
strpos($item_description, '<div id="table">')
);
$item_description = str_replace(
'src="images', 'src="'.$this->uri.'images',
$item_description
);
$item_description = str_replace("\r", '', $item_description);
$item_description = str_replace("\n", '', $item_description);
$item_description = utf8_encode($item_description);
//Build and add final item
$item = new \Item();
$item->uri = $item_uri;
$item->title = $item_name.' '.$item_type.' '.$item_episode;
$item->author = $item_fansub;
$item->timestamp = $item_date;
$item->thumbnailUri = $item_image;
$item->content = $item_description;
$item = array();
$item['uri'] = $item_uri;
$item['title'] = $item_name.' '.$item_type.' '.$item_episode;
$item['author'] = $item_fansub;
$item['timestamp'] = $item_date;
$item['content'] = $item_description;
$this->items[] = $item;
$processedOK++;
//Stop processing once limit is reached
if ($processedOK >= 10)
return;
@ -124,11 +107,12 @@ class AnimeUltimeBridge extends BridgeAbstract {
}
public function getName() {
return 'Latest '.$this->filter.' - Anime-Ultime Bridge';
}
$typeFilter = array_search(
$this->getInput('type'),
$this->parameters[$this->queriedContext]['type']['values']
);
public function getURI() {
return 'http://www.anime-ultime.net/';
return 'Latest '.$typeFilter.' - Anime-Ultime Bridge';
}
public function getCacheDuration() {

66
bridges/ArstechnicaBridge.php Normal file
View File

@ -0,0 +1,66 @@
<?php
#ini_set('display_errors', 'On');
#error_reporting(E_ALL);
class ArstechnicaBridge extends BridgeAbstract {
public $maintainer = "prysme";
public $name = "ArstechnicaBridge";
public $uri = "http://arstechnica.com";
public $description = "The PC enthusiast's resource. Power users and the tools they love, without computing religion";
function StripWithDelimiters($string, $start, $end) {
while (strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
} return $string;
}
function StripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function ExtractContent($url) {
#echo $url;
$html2 = $this->getSimpleHTMLDOM($url);
$text = $html2->find("section[id='article-guts']", 0);
/*foreach ($text->find('<aside id="social-left">') as $node)
{ $node = NULL; }*/
$text = $this->StripWithDelimiters($text->innertext,'<aside id="social-left">','</aside>');
$text = $this->StripWithDelimiters($text,'<figcaption class="caption">','</figcaption>');
$text = $this->StripWithDelimiters($text,'<div class="gallery shortcode-gallery">','</div>');
//error_log("ICI", 0);
//error_log($text, 0);
return $text;
}
public function collectData(){
$html = $this->getSimpleHTMLDOM('http://feeds.arstechnica.com/arstechnica/index') or $this->returnServerError('Could not request NextInpact.');
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 5) {
$item = array();
$item['title'] = $this->StripCDATA($element->find('title', 0)->innertext);
$item['uri'] = $this->StripCDATA($element->find('guid', 0)->plaintext);
$item['author'] = $this->StripCDATA($element->find('author', 0)->innertext);
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $this->ExtractContent($item['uri']);
//$item['content'] = $item['uri'];
$this->items[] = $item;
$limit++;
}
}
}
public function getCacheDuration() {
return 7200; // 2h
}
}

225
bridges/Arte7Bridge.php
View File

@ -1,170 +1,91 @@
<?php
class Arte7Bridge extends BridgeAbstract{
<?php
class Arte7Bridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "mitsukarenai";
public $name = "Arte +7";
public $uri = "http://www.arte.tv/";
public $description = "Returns newest videos from ARTE +7";
public $parameters = array(
'Catégorie (Français)' => array(
'catfr'=>array(
'type'=>'list',
'name'=>'Catégorie',
'values'=>array(
'Toutes les vidéos (français)'=>'toutes-les-videos',
'Actu & société'=>'actu-société',
'Séries & fiction'=>'séries-fiction',
'Cinéma'=>'cinéma',
'Arts & spectacles classiques'=>'arts-spectacles-classiques',
'Culture pop'=>'culture-pop',
'Découverte'=>'découverte',
'Histoire'=>'histoire',
'Junior'=>'junior'
$this->maintainer = "mitsukarenai";
$this->name = "Arte +7";
$this->uri = "http://www.arte.tv/";
$this->description = "Returns newest videos from ARTE +7";
$this->update = "2015-10-31";
$this->parameters["Catégorie (Français)"] =
'[
{
"type" : "list",
"identifier" : "catfr",
"name" : "Catégorie",
"values" : [
{
"name" : "Toutes les vidéos (français)",
"value" : "toutes-les-videos"
},
{
"name" : "Actu & société",
"value" : "actu-société"
},
{
"name" : "Séries & fiction",
"value" : "séries-fiction"
},
{
"name" : "Cinéma",
"value" : "cinéma"
},
{
"name" : "Arts & spectacles classiques",
"value" : "arts-spectacles-classiques"
},
{
"name" : "Culture pop",
"value" : "culture-pop"
},
{
"name" : "Découverte",
"value" : "découverte"
},
{
"name" : "Histoire",
"value" : "histoire"
},
{
"name" : "Junior",
"value" : "junior"
}
)
)
),
'Catégorie (Allemand)' => array(
'catde'=>array(
'type'=>'list',
'name'=>'Catégorie',
'values'=>array(
'Alle Videos (deutsch)'=>'alle-videos',
'Aktuelles & Gesellschaft'=>'aktuelles-gesellschaft',
'Fernsehfilme & Serien'=>'fernsehfilme-serien',
'Kino'=>'kino',
'Kunst & Kultur'=>'kunst-kultur',
'Popkultur & Alternativ'=>'popkultur-alternativ',
'Entdeckung'=>'entdeckung',
'Geschichte'=>'geschichte',
'Junior'=>'junior'
)
)
)
);
]
public function collectData(){
switch($this->queriedContext){
case 'Catégorie (Français)':
$category=$this->getInput('catfr');
$lang='fr';
break;
case 'Catégorie (Allemand)':
$category=$this->getInput('catde');
$lang='de';
break;
}
}
]';
$this->parameters["Catégorie (Allemand)"] =
'[
{
"type" : "list",
"identifier" : "catde",
"name" : "Catégorie",
"values" : [
{
"name" : "Alle Videos (deutsch)",
"value" : "alle-videos"
},
{
"name" : "Aktuelles & Gesellschaft",
"value" : "aktuelles-gesellschaft"
},
{
"name" : "Fernsehfilme & Serien",
"value" : "fernsehfilme-serien"
},
{
"name" : "Kino",
"value" : "kino"
},
{
"name" : "Kunst & Kultur",
"value" : "kunst-kultur"
},
{
"name" : "Popkultur & Alternativ",
"value" : "popkultur-alternativ"
},
{
"name" : "Entdeckung",
"value" : "entdeckung"
},
{
"name" : "Geschichte",
"value" : "geschichte"
},
{
"name" : "Junior",
"value" : "junior"
}
]
}
]';
}
public function collectData(array $param){
function extractVideoset($category='toutes-les-videos', $lang='fr')
{
$url = 'http://www.arte.tv/guide/'.$lang.'/plus7/'.$category;
$input = file_get_contents($url) or die('Could not request ARTE.');
if(strpos($input, 'categoryVideoSet') !== FALSE)
{
$input = explode('categoryVideoSet: ', $input);
$input = explode('}},', $input[1]);
$input = $input[0].'}}';
}
else
{
$input = explode('videoSet: ', $input);
$input = explode('}]},', $input[1]);
$input = $input[0].'}]}';
}
$input = json_decode($input, TRUE);
return $input;
}
$category='toutes-les-videos'; $lang='fr';
if (!empty($param['catfr']))
$category=$param['catfr'];
if (!empty($param['catde']))
{ $category=$param['catde']; $lang='de'; }
$input_json = extractVideoset($category, $lang);
$url = $this->uri.'guide/'.$lang.'/plus7/'.$category;
$input = $this->getContents($url) or die('Could not request ARTE.');
if(strpos($input, 'categoryVideoSet') !== FALSE){
$input = explode('categoryVideoSet: ', $input);
$input = explode('}},', $input[1]);
$input = $input[0].'}}';
}else{
$input = explode('videoSet: ', $input);
$input = explode('}]},', $input[1]);
$input = $input[0].'}]}';
}
$input_json = json_decode($input, TRUE);
foreach($input_json['videos'] as $element) {
$item = new \Item();
$item->uri = str_replace("autoplay=1", "", $element['url']);
$item->id = $element['id'];
$item = array();
$item['uri'] = str_replace("autoplay=1", "", $element['url']);
$item['id'] = $element['id'];
$hack_broadcast_time = $element['rights_end'];
$hack_broadcast_time = strtok($hack_broadcast_time, 'T');
$hack_broadcast_time = strtok('T');
$item->timestamp = strtotime($element['scheduled_on'].'T'.$hack_broadcast_time);
$item->thumbnailUri = $element['thumbnail_url'];
$item->title = $element['title'];
$item['timestamp'] = strtotime($element['scheduled_on'].'T'.$hack_broadcast_time);
$item['title'] = $element['title'];
if (!empty($element['subtitle']))
$item->title = $element['title'].' | '.$element['subtitle'];
$item->duration = round((int)$element['duration']/60);
$item->content = $element['teaser'].'<br><br>'.$item->duration.'min<br><a href="'.$item->uri.'"><img src="' . $item->thumbnailUri . '" /></a>';
$item['title'] = $element['title'].' | '.$element['subtitle'];
$item['duration'] = round((int)$element['duration']/60);
$item['content'] = $element['teaser'].'<br><br>'.$item['duration'].'min<br><a href="'.$item['uri'].'"><img src="' . $element['thumbnail_url'] . '" /></a>';
$this->items[] = $item;
}
}
public function getName(){
return 'Arte7';
}
public function getURI(){
return 'http://www.arte.tv/';
}
public function getCacheDuration(){
return 1800; // 30 minutes
}

56
bridges/AskfmBridge.php Normal file
View File

@ -0,0 +1,56 @@
<?php
class AskfmBridge extends BridgeAbstract{
public $maintainer = "az5he6ch";
public $name = "Ask.fm Answers";
public $uri = "http://ask.fm/";
public $description = "Returns answers from an Ask.fm user";
public $parameters = array(
'Ask.fm username'=>array(
'u'=>array(
'name'=>'Username',
'required'=>true
)
)
);
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->getURI())
or $this->returnServerError('Requested username can\'t be found.');
foreach($html->find('div.streamItem-answer') as $element) {
$item = array();
$item['uri'] = $this->uri.$element->find('a.streamItemsAge',0)->href;
$question = trim($element->find('h1.streamItemContent-question',0)->innertext);
$item['title'] = trim(htmlspecialchars_decode($element->find('h1.streamItemContent-question',0)->plaintext, ENT_QUOTES));
$answer = trim($element->find('p.streamItemContent-answer',0)->innertext);
#$item['update'] = $element->find('a.streamitemsage',0)->data-hint; // Doesn't work, DOM parser doesn't seem to like data-hint, dunno why
$visual = $element->find('div.streamItemContent-visual',0)->innertext; // This probably should be cleaned up, especially for YouTube embeds
//Fix tracking links, also doesn't work
foreach($element->find('a') as $link) {
if (strpos($link->href, 'l.ask.fm') !== false) {
#$link->href = str_replace('#_=_', '', get_headers($link->href, 1)['Location']); // Too slow
$link->href = $link->plaintext;
}
}
$content = '<p>' . $question . '</p><p>' . $answer . '</p><p>' . $visual . '</p>';
// Fix relative links without breaking // scheme used by YouTube stuff
$content = preg_replace('#href="\/(?!\/)#', 'href="'.$this->uri,$content);
$item['content'] = $content;
$this->items[] = $item;
}
}
public function getName(){
return $this->name.' : '.$this->getInput('u');
}
public function getURI(){
return $this->uri.urlencode($this->getInput('u')).'/answers/more?page=0';
}
public function getCacheDuration(){
return 300; // 5 minutes
}
}

63
bridges/BandcampBridge.php
View File

@ -1,58 +1,43 @@
<?php
class BandcampBridge extends BridgeAbstract{
private $request;
public $maintainer = "sebsauvage";
public $name = "Bandcamp Tag";
public $uri = "http://bandcamp.com/";
public $description = "New bandcamp release by tag";
public $parameters = array( array(
'tag'=>array(
'name'=>'tag',
'type'=>'text',
'required'=>true
)
));
public function loadMetadatas() {
$this->maintainer = "sebsauvage";
$this->name = "Bandcamp Tag";
$this->uri = "http://bandcamp.com/";
$this->description = "New bandcamp release by tag";
$this->update = "2014-05-25";
$this->parameters[] =
'[
{
"name" : "tag",
"type" : "text",
"identifier" : "tag"
}
]';
}
public function collectData(array $param){
$html = '';
if (isset($param['tag'])) {
$this->request = $param['tag'];
$html = $this->file_get_html('http://bandcamp.com/tag/'.urlencode($this->request).'?sort_field=date') or $this->returnError('No results for this query.', 404);
}
else {
$this->returnError('You must specify tag (/tag/...)', 400);
}
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->getURI())
or $this->returnServerError('No results for this query.');
foreach($html->find('li.item') as $release) {
$script = $release->find('div.art', 0)->getAttribute('onclick');
$uri = ltrim($script, "return 'url(");
$uri = rtrim($uri, "')");
$item = new \Item();
$item->name = $release->find('div.itemsubtext',0)->plaintext . ' - ' . $release->find('div.itemtext',0)->plaintext;
$item->title = $release->find('div.itemsubtext',0)->plaintext . ' - ' . $release->find('div.itemtext',0)->plaintext;
$item->content = '<img src="' . $uri . '"/><br/>' . $release->find('div.itemsubtext',0)->plaintext . ' - ' . $release->find('div.itemtext',0)->plaintext;
$item->id = $release->find('a',0)->getAttribute('href');
$item->uri = $release->find('a',0)->getAttribute('href');
$item = array();
$item['author'] = $release->find('div.itemsubtext',0)->plaintext . ' - ' . $release->find('div.itemtext',0)->plaintext;
$item['title'] = $release->find('div.itemsubtext',0)->plaintext . ' - ' . $release->find('div.itemtext',0)->plaintext;
$item['content'] = '<img src="' . $uri . '"/><br/>' . $release->find('div.itemsubtext',0)->plaintext . ' - ' . $release->find('div.itemtext',0)->plaintext;
$item['id'] = $release->find('a',0)->getAttribute('href');
$item['uri'] = $release->find('a',0)->getAttribute('href');
$this->items[] = $item;
}
}
public function getName(){
return (!empty($this->request) ? $this->request .' - ' : '') .'Bandcamp Tag';
public function getURI(){
return $this->uri.'tag/'.urlencode($this->getInput('tag')).'?sort_field=date';
}
public function getURI(){
return 'http://bandcamp.com';
public function getName(){
return $this->getInput('tag') .' - '.'Bandcamp Tag';
}
public function getCacheDuration(){

65
bridges/BastaBridge.php
View File

@ -1,52 +1,35 @@
<?php
class BastaBridge extends BridgeAbstract{
public $maintainer = "qwertygc";
public $name = "Bastamag Bridge";
public $uri = "http://www.bastamag.net/";
public $description = "Returns the newest articles.";
public function loadMetadatas() {
$this->maintainer = "qwertygc";
$this->name = "Bastamag Bridge";
$this->uri = "http://www.bastamag.net/";
$this->description = "Returns the newest articles.";
$this->update = "2014-05-25";
public function collectData(){
// Replaces all relative image URLs by absolute URLs. Relative URLs always start with 'local/'!
function ReplaceImageUrl($content){
return preg_replace('/src=["\']{1}([^"\']+)/ims', 'src=\''.$this->uri.'$1\'', $content);
}
public function collectData(array $param){
function BastaExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = $html2->find('div.texte', 0)->innertext;
return $text;
}
$html = $this->file_get_html('http://www.bastamag.net/spip.php?page=backend') or $this->returnError('Could not request Bastamag.', 404);
$html = $this->getSimpleHTMLDOM($this->uri.'spip.php?page=backend')
or $this->returnServerError('Could not request Bastamag.');
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 10) {
$item = new \Item();
$item->title = $element->find('title', 0)->innertext;
$item->uri = $element->find('guid', 0)->plaintext;
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = BastaExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
if($limit < 10) {
$item = array();
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = $element->find('guid', 0)->plaintext;
$item['timestamp'] = strtotime($element->find('dc:date', 0)->plaintext);
$item['content'] = ReplaceImageUrl($this->getSimpleHTMLDOM($item['uri'])->find('div.texte', 0)->innertext);
$this->items[] = $item;
$limit++;
}
}
}
}
public function getName(){
return 'Bastamag Bridge';
}
public function getURI(){
return 'http://bastamag.net/';
}
public function getCacheDuration(){
return 3600*2; // 2 hours
// return 0; // 2 hours
}
public function getCacheDuration(){
return 3600*2; // 2 hours
}
}
?>

40
bridges/BlaguesDeMerdeBridge.php
View File

@ -1,50 +1,36 @@
<?php
class BlaguesDeMerdeBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "superbaillot.net";
public $name = "Blagues De Merde";
public $uri = "http://www.blaguesdemerde.fr/";
public $description = "Blagues De Merde";
$this->maintainer = "superbaillot.net";
$this->name = "Blagues De Merde";
$this->uri = "http://www.blaguesdemerde.fr/";
$this->description = "Blagues De Merde";
$this->update = "16/10/2013";
}
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Could not request BDM.');
public function collectData(array $param){
$html = $this->file_get_html('http://www.blaguesdemerde.fr/') or $this->returnError('Could not request BDM.', 404);
foreach($html->find('article.joke_contener') as $element) {
$item = new Item();
$item = array();
$temp = $element->find('a');
if(isset($temp[2]))
{
$item->content = trim($element->find('div.joke_text_contener', 0)->innertext);
$item['content'] = trim($element->find('div.joke_text_contener', 0)->innertext);
$uri = $temp[2]->href;
$item->uri = $uri;
$item->title = substr($uri, (strrpos($uri, "/") + 1));
$item['uri'] = $uri;
$item['title'] = substr($uri, (strrpos($uri, "/") + 1));
$date = $element->find("li.bdm_date",0)->innertext;
$time = mktime(0, 0, 0, substr($date, 3, 2), substr($date, 0, 2), substr($date, 6, 4));
$item->timestamp = $time;
$item->name = $element->find("li.bdm_pseudo",0)->innertext;;
$item['timestamp'] = $time;
$item['author'] = $element->find("li.bdm_pseudo",0)->innertext;;
$this->items[] = $item;
}
}
}
public function getName(){
return 'blaguesdemerde';
}
public function getURI(){
return 'http://www.blaguesdemerde.fr/';
}
public function getCacheDuration(){
return 7200; // 2h hours
}
public function getDescription(){
return "Blagues De Merde via rss-bridge";
}
}
?>

89
bridges/BooruprojectBridge.php
View File

@ -1,72 +1,47 @@
<?php
class BooruprojectBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "mitsukarenai";
public $name = "Booruproject";
public $uri = "http://booru.org/";
public $description = "Returns images from given page and booruproject instance (****.booru.org)";
$this->maintainer = "mitsukarenai";
$this->name = "Booruproject";
$this->uri = "http://booru.org/";
$this->description = "Returns images from given page and booruproject instance (****.booru.org)";
$this->update = "2015-09-12";
public $parameters = array( array(
'i'=>array(
'name'=>'instance (required)',
'required'=>true
),
'p'=>array(
'name'=>'page',
'type'=>'number'
),
't'=>array('name'=>'tags')
));
$this->parameters[] =
'[
{
"name" : "instance (required)",
"required" : "true",
"identifier" : "i"
},
{
"name" : "page",
"identifier" : "p"
},
{
"name" : "tags",
"identifier" : "t"
}
function getURI(){
return 'http://'.$this->getInput('i').'.booru.org/';
}
]';
}
public function collectData(array $param){
$page = 0; $tags = '';
if (!empty($param['p'])) {
$page = (int)preg_replace("/[^0-9]/",'', $param['p']);
$page = $page - 1;
$page = $page * 20;
}
if (!empty($param['t'])) {
$tags = '&tags='.urlencode($param['t']);
}
if (empty($param['i'])) {
$this->returnError('Please enter a ***.booru.org instance.', 404);
}
$html = $this->file_get_html("http://".$param['i'].".booru.org/index.php?page=post&s=list&pid=".$page.$tags) or $this->returnError('Could not request Booruproject.', 404);
public function collectData(){
$html = $this->getSimpleHTMLDOM(
$this->getURI().'index.php?page=post&s=list'
.'&pid='.($this->getInput('p')?($this->getInput('p') -1)*20:'')
.'&tags='.urlencode($this->getInput('t'))
) or $this->returnServerError('Could not request Booruprojec.');
foreach($html->find('div[class=content] span') as $element) {
$item = new \Item();
$item->uri = 'http://'.$param['i'].'.booru.org/'.$element->find('a', 0)->href;
$item->postid = (int)preg_replace("/[^0-9]/",'', $element->find('a', 0)->getAttribute('id'));
$item->timestamp = time();
$item->thumbnailUri = $element->find('img', 0)->src;
$item->tags = $element->find('img', 0)->getAttribute('title');
$item->title = 'Booruproject '.$param['i'].' | '.$item->postid;
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" /></a><br>Tags: '.$item->tags;
$this->items[] = $item;
$item = array();
$item['uri'] = $this->getURI().$element->find('a', 0)->href;
$item['postid'] = (int)preg_replace("/[^0-9]/",'', $element->find('a', 0)->getAttribute('id'));
$item['timestamp'] = time();
$item['tags'] = $element->find('img', 0)->getAttribute('title');
$item['title'] = 'Booruproject '.$this->getInput('i').' | '.$item['postid'];
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $element->find('img', 0)->src . '" /></a><br>Tags: '.$item['tags'];
$this->items[] = $item;
}
}
public function getName(){
return 'Booruproject';
}
public function getURI(){
return 'http://booru.org/';
}
public function getCacheDuration(){
return 1800; // 30 minutes
}

99
bridges/CADBridge.php
View File

@ -1,69 +1,66 @@
<?php
class CADBridge extends BridgeAbstract{
public $maintainer = "nyutag";
public $name = "CAD Bridge";
public $uri = "http://www.cad-comic.com/";
public $description = "Returns the newest articles.";
public function loadMetadatas() {
private function CADExtractContent($url) {
$html3 = $this->getSimpleHTMLDOM($url);
$this->maintainer = "nyutag";
$this->name = "CAD Bridge";
$this->uri = "http://www.cad-comic.com/";
$this->description = "Returns the newest articles.";
$this->update = "2015-04-03";
// The request might fail due to missing https support or wrong URL
if($html3 == false)
return 'Daily comic not released yet';
}
public function collectData(array $param){
function CADUrl($string) {
$html2 = explode("\"", $string);
$string = $html2[1];
if (substr($string,0,4) != 'http')
return 'notanurl';
return $string;
}
function CADExtractContent($url) {
$html3 = $this->file_get_html($url);
$htmlpart = explode("/", $url);
if ($htmlpart[3] == 'cad')
preg_match_all("/http:\/\/cdn2\.cad-comic\.com\/comics\/cad-\S*png/", $html3, $url2);
if ($htmlpart[3] == 'sillies')
preg_match_all("/http:\/\/cdn2\.cad-comic\.com\/comics\/sillies-\S*gif/", $html3, $url2);
switch ($htmlpart[3]){
case 'cad':
preg_match_all("/http:\/\/cdn2\.cad-comic\.com\/comics\/cad-\S*png/", $html3, $url2);
break;
case 'sillies':
preg_match_all("/http:\/\/cdn2\.cad-comic\.com\/comics\/sillies-\S*gif/", $html3, $url2);
break;
default:
return 'Daily comic not released yet';
}
$img = implode ($url2[0]);
$html3->clear();
unset ($html3);
if ($img == '')
return 'Daily comic not realease yet';
return 'Daily comic not released yet';
return '<img src="'.$img.'"/>';
}
public function collectData(){
function CADUrl($string) {
$html2 = explode("\"", $string);
$string = $html2[1];
if (substr($string,0,4) != 'http')
return 'notanurl';
return $string;
}
$html = $this->file_get_html('http://cdn2.cad-comic.com/rss.xml') or $this->returnError('Could not request CAD.', 404);
$html = $this->getSimpleHTMLDOM('http://cdn2.cad-comic.com/rss.xml') or $this->returnServerError('Could not request CAD.');
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 5) {
$item = new \Item();
$item->title = $element->find('title', 0)->innertext;
$item->uri = CADUrl($element->find('description', 0)->innertext);
if ($item->uri != 'notanurl') {
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = CADExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
}
if($limit < 5) {
$item = array();
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = CADUrl($element->find('description', 0)->innertext);
if ($item['uri'] != 'notanurl') {
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $this->CADExtractContent($item['uri']);
$this->items[] = $item;
$limit++;
}
}
}
}
}
public function getName(){
return 'CAD Bridge';
}
public function getURI(){
return 'http://www.cad-comic.com/';
}
public function getCacheDuration(){
return 3600*2; // 2 hours
// return 0;
}
public function getCacheDuration(){
return 3600*2; // 2 hours
}
}
?>

67
bridges/CNETBridge.php
View File

@ -1,26 +1,16 @@
<?php
class CNETBridge extends BridgeAbstract {
private $topicName = '';
public $maintainer = 'ORelio';
public $name = 'CNET News';
public $uri = 'http://www.cnet.com/';
public $description = 'Returns the newest articles. <br /> You may specify a topic found in some section URLs, else all topics are selected.';
public function loadMetadatas() {
public $parameters = array( array(
'topic'=>array('name'=>'Topic name')
));
$this->maintainer = 'ORelio';
$this->name = 'CNET News';
$this->uri = 'http://www.cnet.com/';
$this->description = 'Returns the newest articles. <br /> You may specify a topic found in some section URLs, else all topics are selected.';
$this->update = '2016-03-16';
$this->parameters[] =
'[
{
"name" : "Topic name",
"identifier" : "topic"
}
]';
}
public function collectData(array $param) {
public function collectData(){
function ExtractFromDelimiters($string, $start, $end) {
if (strpos($string, $start) !== false) {
@ -47,44 +37,30 @@ class CNETBridge extends BridgeAbstract {
return $article_html;
}
if (!empty($param['topic']))
$this->topicName = $param['topic'];
$pageUrl = 'http://www.cnet.com/'.(empty($this->topicName) ? '' : 'topics/'.$this->topicName.'/');
$html = $this->file_get_html($pageUrl) or $this->returnError('Could not request CNET: '.$pageUrl, 500);
$pageUrl = $this->uri.(empty($this->getInput('topic')) ? '' : 'topics/'.$this->getInput('topic').'/');
$html = $this->getSimpleHTMLDOM($pageUrl) or $this->returnServerError('Could not request CNET: '.$pageUrl);
$limit = 0;
foreach($html->find('div.assetBody') as $element) {
if ($limit < 8) {
$article_title = trim($element->find('h2', 0)->plaintext);
$article_uri = 'http://www.cnet.com'.($element->find('a', 0)->href);
$article_thumbnail = $element->parent()->find('img', 0)->src;
$article_uri = $this->uri.($element->find('a', 0)->href);
$article_timestamp = strtotime($element->find('time.assetTime', 0)->plaintext);
$article_author = trim($element->find('a[rel=author]', 0)->plaintext);
if (!empty($article_title) && !empty($article_uri) && strpos($article_uri, '/news/') !== false) {
$article_html = $this->file_get_html($article_uri) or $this->returnError('Could not request CNET: '.$article_uri, 500);
if (is_null($article_thumbnail))
$article_thumbnail = $article_html->find('div.originalImage', 0);
if (is_null($article_thumbnail))
$article_thumbnail = $article_html->find('span.imageContainer', 0);
if (is_object($article_thumbnail))
$article_thumbnail = $article_thumbnail->find('img', 0)->src;
$article_html = $this->getSimpleHTMLDOM($article_uri) or $this->returnServerError('Could not request CNET: '.$article_uri);
$article_content = trim(CleanArticle(ExtractFromDelimiters($article_html, '<div class="articleContent', '<footer>')));
$item = new \Item();
$item->uri = $article_uri;
$item->thumbnailUri = $article_thumbnail;
$item->title = $article_title;
$item->author = $article_author;
$item->timestamp = $article_timestamp;
$item->content = $article_content;
$item = array();
$item['uri'] = $article_uri;
$item['title'] = $article_title;
$item['author'] = $article_author;
$item['timestamp'] = $article_timestamp;
$item['content'] = $article_content;
$this->items[] = $item;
$limit++;
}
@ -93,15 +69,10 @@ class CNETBridge extends BridgeAbstract {
}
public function getName() {
return 'CNET News Bridge'.(empty($this->topicName) ? '' : ' - '.$this->topicName);
}
public function getURI() {
return 'http://www.cnet.com/';
return 'CNET News Bridge'.(empty($this->getInput('topic')) ? '' : ' - '.$this->getInput('topic'));
}
public function getCacheDuration() {
return 1800; // 30 minutes
// return 0;
}
}

114
bridges/CastorusBridge.php Normal file
View File

@ -0,0 +1,114 @@
<?php
class CastorusBridge extends BridgeAbstract {
public $maintainer = "logmanoriginal";
public $name = "Castorus Bridge";
public $uri = 'http://www.castorus.com';
public $description = "Returns the latest changes";
public $parameters = array(
'Get latest changes' => array(),
'Get latest changes via ZIP code' => array(
'zip'=>array(
'name'=>'ZIP code',
'type'=>'text',
'required'=>true,
'exampleValue'=>'74910, 74',
'title'=>'Insert ZIP code (complete or partial)'
)
),
'Get latest changes via city name' => array(
'city'=>array(
'name'=>'City name',
'type'=>'text',
'required'=>true,
'exampleValue'=>'Seyssel, Seys',
'title'=>'Insert city name (complete or partial)'
)
)
);
// Extracts the tile from an actitiy
private function ExtractActivityTitle($activity){
$title = $activity->find('a', 0);
if(!$title)
$this->returnServerError('Cannot find title!');
return htmlspecialchars(trim($title->plaintext));
}
// Extracts the url from an actitiy
private function ExtractActivityUrl($activity){
$url = $activity->find('a', 0);
if(!$url)
$this->returnServerError('Cannot find url!');
return $this->uri . $url->href;
}
// Extracts the time from an activity
private function ExtractActivityTime($activity){
// Unfortunately the time is part of the parent node,
// so we have to clear all child nodes first
$nodes = $activity->find('*');
if(!$nodes)
$this->returnServerError('Cannot find nodes!');
foreach($nodes as $node){
$node->outertext = '';
}
return strtotime($activity->innertext);
}
// Extracts the price change
private function ExtractActivityPrice($activity){
$price = $activity->find('span', 1);
if(!$price)
$this->returnServerError('Cannot find price!');
return $price->innertext;
}
public function collectData(){
$zip_filter = trim($this->getInput('zip'));
$city_filter = trim($this->getInput('city'));
$html = $this->getSimpleHTMLDOM($this->uri);
if(!$html)
$this->returnServerError('Could not load data from ' . $this->uri . '!');
$activities = $html->find('div#activite/li');
if(!$activities)
$this->returnServerError('Failed to find activities!');
foreach($activities as $activity){
$item = array();
$item['title'] = $this->ExtractActivityTitle($activity);
$item['uri'] = $this->ExtractActivityUrl($activity);
$item['timestamp'] = $this->ExtractActivityTime($activity);
$item['content'] = '<a href="' . $item['uri'] . '">' . $item['title'] . '</a><br><p>'
. $this->ExtractActivityPrice($activity) . '</p>';
if(isset($zip_filter) && !(substr($item['title'], 0, strlen($zip_filter)) === $zip_filter)){
continue; // Skip this item
}
if(isset($city_filter) && !(substr($item['title'], strpos($item['title'], ' ') + 1, strlen($city_filter)) === $city_filter)){
continue; // Skip this item
}
$this->items[] = $item;
}
}
public function getCacheDuration(){
return 600; // 10 minutes
}
}

56
bridges/CoinDeskBridge.php
View File

@ -1,56 +0,0 @@
<?php
class CoinDeskBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "mitsukarenai";
$this->name = "CoinDesk";
$this->uri = "http://www.coindesk.com/";
$this->description = "Returns the 5 newest posts from CoinDesk (full text)";
$this->update = "2014-05-30";
}
public function collectData(array $param){
function CoinDeskStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function CoinDeskExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = $html2->find('div.single-content', 0)->innertext;
$text = strip_tags($text, '<p><a><img>');
return $text;
}
$html = $this->file_get_html('http://www.coindesk.com/feed/atom/') or $this->returnError('Could not request CoinDesk.', 404);
$limit = 0;
foreach($html->find('entry') as $element) {
if($limit < 5) {
$item = new \Item();
$item->title = CoinDeskStripCDATA($element->find('title', 0)->innertext);
$item->author = $element->find('author', 0)->plaintext;
$item->uri = $element->find('link', 0)->href;
$item->timestamp = strtotime($element->find('published', 0)->plaintext);
$item->content = CoinDeskExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'CoinDesk';
}
public function getURI(){
return 'http://www.coindesk.com/';
}
public function getCacheDuration(){
return 1800; // 30min
}
}

51
bridges/CollegeDeFranceBridge.php
View File

@ -1,15 +1,12 @@
<?php
class CollegeDeFranceBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "pit-fgfjiudghdf";
$this->name = "CollegeDeFrance";
$this->uri = "http://www.college-de-france.fr/";
$this->description = "Returns the latest audio and video from CollegeDeFrance";
$this->update = "2016-05-01";
}
public $maintainer = "pit-fgfjiudghdf";
public $name = "CollegeDeFrance";
public $uri = "http://www.college-de-france.fr/";
public $description = "Returns the latest audio and video from CollegeDeFrance";
public function collectData(array $param) {
public function collectData(){
$months = array(
'01' => 'janv.',
'02' => 'févr.',
@ -33,10 +30,11 @@ class CollegeDeFranceBridge extends BridgeAbstract{
* </a>
* </li>
*/
$html = $this->file_get_html('http://www.college-de-france.fr/components/search-audiovideo.jsp?fulltext=&siteid=1156951719600&lang=FR&type=all') or $this->returnError('Could not request CollegeDeFrance.', 404);
$html = $this->getSimpleHTMLDOM($this->uri.'components/search-audiovideo.jsp?fulltext=&siteid=1156951719600&lang=FR&type=all')
or $this->returnServerError('Could not request CollegeDeFrance.');
foreach($html->find('a[data-target]') as $element) {
$item = new \Item();
$item->title = $element->find('.title', 0)->plaintext;
$item = array();
$item['title'] = $element->find('.title', 0)->plaintext;
// Most relative URLs contains an hour in addition to the date, so let's use it
// <a href="/site/yann-lecun/course-2016-04-08-11h00.htm" data-target="after">
//
@ -46,22 +44,29 @@ class CollegeDeFranceBridge extends BridgeAbstract{
// <a href="/site/institut-physique/The-Mysteries-of-Decoherence-Sebastien-Gleyzes-[Video-3-35].htm" data-target="after">
$timezone = new DateTimeZone('Europe/Paris');
// strpos($element->href, '201') will break in 2020 but it'll probably break prior to then due to site changes anyway
$d = DateTime::createFromFormat('!Y-m-d-H\hi', substr($element->href, strpos($element->href, '201'), 16), $timezone) ?: DateTime::createFromFormat('!d m Y', trim(str_replace(array_values($months), array_keys($months), $element->find('.date', 0)->plaintext)), $timezone);
$item->timestamp = $d->format('U');
$item->content = $element->find('.lecturer', 0)->innertext . ' - ' . $element->find('.title', 0)->innertext;
$item->uri = 'http://www.college-de-france.fr' . $element->href;
$d = DateTime::createFromFormat(
'!Y-m-d-H\hi',
substr($element->href, strpos($element->href, '201'), 16),
$timezone
);
if(!$d){
$d=DateTime::createFromFormat(
'!d m Y',
trim(str_replace(
array_values($months),
array_keys($months),
$element->find('.date', 0)->plaintext
)),
$timezone
);
}
$item['timestamp'] = $d->format('U');
$item['content'] = $element->find('.lecturer', 0)->innertext . ' - ' . $element->find('.title', 0)->innertext;
$item['uri'] = $this->uri . $element->href;
$this->items[] = $item;
}
}
public function getName(){
return 'CollegeDeFrance';
}
public function getURI(){
return 'http://www.college-de-france.fr/';
}
public function getCacheDuration(){
return 3600*3; // 3 hours
}

69
bridges/CommonDreamsBridge.php
View File

@ -1,58 +1,39 @@
<?php
class CommonDreamsBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "nyutag";
public $name = "CommonDreams Bridge";
public $uri = "http://www.commondreams.org/";
public $description = "Returns the newest articles.";
$this->maintainer = "nyutag";
$this->name = "CommonDreams Bridge";
$this->uri = "http://www.commondreams.org/";
$this->description = "Returns the newest articles.";
$this->update = "2015-04-03";
}
public function collectData(array $param){
function CommonDreamsUrl($string) {
$html2 = explode(" ", $string);
$string = $html2[2] . "/node/" . $html2[0];
return $string;
}
function CommonDreamsExtractContent($url) {
$html3 = $this->file_get_html($url);
private function CommonDreamsExtractContent($url) {
$html3 = $this->getSimpleHTMLDOM($url);
$text = $html3->find('div[class=field--type-text-with-summary]', 0)->innertext;
$html3->clear();
unset ($html3);
return $text;
}
public function collectData(){
function CommonDreamsUrl($string) {
$html2 = explode(" ", $string);
$string = $html2[2] . "/node/" . $html2[0];
return $string;
}
$html = $this->file_get_html('http://www.commondreams.org/rss.xml') or $this->returnError('Could not request CommonDreams.', 404);
$html = $this->getSimpleHTMLDOM('http://www.commondreams.org/rss.xml') or $this->returnServerError('Could not request CommonDreams.');
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 4) {
$item = new \Item();
$item->title = $element->find('title', 0)->innertext;
$item->uri = CommonDreamsUrl($element->find('guid', 0)->innertext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = CommonDreamsExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
if($limit < 4) {
$item = array();
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = CommonDreamsUrl($element->find('guid', 0)->innertext);
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $this->CommonDreamsExtractContent($item['uri']);
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'CommonDreams Bridge';
}
public function getURI(){
return 'http://www.commondreams.org/';
}
public function getCacheDuration(){
return 3600; // 1 hours
// return 0;
}
}
}

52
bridges/CopieDoubleBridge.php
View File

@ -1,62 +1,42 @@
<?php
class CopieDoubleBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "superbaillot.net";
public $name = "CopieDouble";
public $uri = "http://www.copie-double.com/";
public $description = "CopieDouble";
$this->maintainer = "superbaillot.net";
$this->name = "CopieDouble";
$this->uri = "http://www.copie-double.com/";
$this->description = "CopieDouble";
$this->update = "12/12/2013";
}
public function collectData(array $param){
$html = $this->file_get_html('http://www.copie-double.com/') or $this->returnError('Could not request CopieDouble.', 404);
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Could not request CopieDouble.');
$table = $html->find('table table', 2);
foreach($table->find('tr') as $element)
{
$td = $element->find('td', 0);
$cpt++;
if($td->class == "couleur_1")
{
$item = new Item();
$item = array();
$title = $td->innertext;
$pos = strpos($title, "<a");
$title = substr($title, 0, $pos);
$item->title = $title;
$item['title'] = $title;
}
elseif(strpos($element->innertext, "/images/suivant.gif") === false)
{
$a=$element->find("a", 0);
$item->uri = "http://www.copie-double.com" . $a->href;
$content = str_replace('src="/', 'src="http://www.copie-double.com/',$element->find("td", 0)->innertext);
$content = str_replace('href="/', 'href="http://www.copie-double.com/',$content);
$item->content = $content;
$item['uri'] = $this->uri . $a->href;
$content = str_replace('src="/', 'src="/'.$this->uri,$element->find("td", 0)->innertext);
$content = str_replace('href="/', 'href="'.$this->uri,$content);
$item['content'] = $content;
$this->items[] = $item;
}
}
}
public function getName(){
return 'CopieDouble';
}
public function getURI(){
return 'http://www.copie-double.com';
}
public function getDescription(){
return 'CopieDouble via rss-bridge';
}
public function getCacheDuration(){
return 14400; // 4 hours
}
}
?>

70
bridges/CourrierInternationalBridge.php
View File

@ -1,74 +1,58 @@
<?php
class CourrierInternationalBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "teromene";
public $name = "Courrier International Bridge";
public $uri = "http://CourrierInternational.com/";
public $description = "Courrier International bridge";
$this->maintainer = "teromene";
$this->name = "CourrierInternational";
$this->uri = "http://CourrierInternational.fr/";
$this->description = "Courrier International bridge";
$this->update = "01/09/2015";
public function collectData(){
}
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Error.');
public function collectData(array $param){
$html = '';
$html = $this->file_get_html('http://www.courrierinternational.com/') or $this->returnError('Error.', 500);
$element = $html->find("article");
$article_count = 1;
$article_count = 1;
foreach($element as $article) {
$item = new \Item();
$item->uri = $article->parent->getAttribute("href");
if(strpos($item->uri, "http") === FALSE) {
$item->uri = "http://courrierinternational.fr/".$item->uri;
$item = array();
$item['uri'] = $article->parent->getAttribute("href");
if(strpos($item['uri'], "http") === FALSE) {
$item['uri'] = $this->uri.$item['uri'];
}
$page = $this->file_get_html($item->uri);
$page = $this->getSimpleHTMLDOM($item['uri']);
$cleaner = new HTMLSanitizer();
$item->content = $cleaner->sanitize($page->find("div.article-text")[0]);
$item->title = strip_tags($article->find(".title")[0]);
$item['content'] = $cleaner->sanitize($page->find("div.article-text")[0]);
$item['title'] = strip_tags($article->find(".title")[0]);
$dateTime = date_parse($page->find("time")[0]);
$item->timestamp = mktime(
$dateTime['hour'],
$dateTime['minute'],
$dateTime['second'],
$dateTime['month'],
$dateTime['day'],
$item['timestamp'] = mktime(
$dateTime['hour'],
$dateTime['minute'],
$dateTime['second'],
$dateTime['month'],
$dateTime['day'],
$dateTime['year']
);
$this->items[] = $item;
$article_count ++;
if($article_count > 5) break;
}
}
public function getName(){
return 'Courrier International Bridge';
}
public function getURI(){
return 'http://courrierinternational.com';
}
public function getCacheDuration(){
return 300; // 5 minutes
}

98
bridges/CpasbienBridge.php
View File

@ -1,90 +1,54 @@
<?php
// simple_html_dom funtion to get the dom from contents instead from file
function content_get_html($contents, $maxLen=-1, $lowercase = true, $forceTagsClosed=true, $target_charset = DEFAULT_TARGET_CHARSET, $stripRN=true, $defaultBRText=DEFAULT_BR_TEXT, $defaultSpanText=DEFAULT_SPAN_TEXT)
{
// We DO force the tags to be terminated.
$dom = new simple_html_dom(null, $lowercase, $forceTagsClosed, $target_charset, $stripRN, $defaultBRText, $defaultSpanText);
if (empty($contents) || strlen($contents) > MAX_FILE_SIZE)
{
return false;
}
// The second parameter can force the selectors to all be lowercase.
$dom->load($contents, $lowercase, $stripRN);
return $dom;
}
class CpasbienBridge extends HttpCachingBridgeAbstract{
private $request;
public function loadMetadatas() {
public $maintainer = "lagaisse";
public $name = "Cpasbien Bridge";
public $uri = "http://www.cpasbien.io";
public $description = "Returns latest torrents from a request query";
$this->maintainer = "lagaisse";
$this->name = "Cpasbien Bridge";
$this->uri = "http://www.cpasbien.io";
$this->description = "Returns latest torrents from a request query";
$this->update = "2016-01-26";
public $parameters = array( array(
'q'=>array(
'name'=>'Search',
'required'=>true,
'title'=>'Type your search'
)
));
$this->parameters[] =
'[
{
"name" : "Search",
"identifier" : "q",
"required" : true,
"title" : "Type your search"
}
]';
}
public function collectData(array $param){
$this->loadMetadatas();
$html = '';
if (isset($param['q'])) { /* keyword search mode */
$this->request = str_replace(" ","-",trim($param['q']));
$html = $this->file_get_html($this->uri.'/recherche/'.urlencode($this->request).'.html') or $this->returnError('No results for this query.', 404);
}
else {
$this->returnError('You must specify a keyword (?q=...).', 400);
}
public function collectData(){
$request = str_replace(" ","-",trim($this->getInput('q')));
$html = $this->getSimpleHTMLDOM($this->uri.'/recherche/'.urlencode($request).'.html')
or $this->returnServerError('No results for this query.');
foreach ($html->find('#gauche',0)->find('div') as $episode) {
if ($episode->getAttribute('class')=='ligne0' || $episode->getAttribute('class')=='ligne1')
if ($episode->getAttribute('class')=='ligne0' ||
$episode->getAttribute('class')=='ligne1')
{
$htmlepisode=content_get_html($this->get_cached($episode->find('a', 0)->getAttribute('href')));
$htmlepisode=$this->get_cached($episode->find('a', 0)->getAttribute('href'));
$item = new \Item();
$item->name = $episode->find('a', 0)->text();
$item->title = $episode->find('a', 0)->text();
$item->timestamp = $this->get_cached_time($episode->find('a', 0)->getAttribute('href'));
$item = array();
$item['author'] = $episode->find('a', 0)->text();
$item['title'] = $episode->find('a', 0)->text();
$item['timestamp'] = $this->get_cached_time($episode->find('a', 0)->getAttribute('href'));
$textefiche=$htmlepisode->find('#textefiche', 0)->find('p',1);
if (isset($textefiche)) {
$item->content = $textefiche->text();
}
else {
$item->content = $htmlepisode->find('#textefiche', 0)->find('p',0)->text();
$item['content'] = $textefiche->text();
} else {
$p=$htmlepisode->find('#textefiche',0)->find('p');
if(!empty($p)){
$item['content'] = $htmlepisode->find('#textefiche', 0)->find('p',0)->text();
}
}
$item->id = $episode->find('a', 0)->getAttribute('href');
$item->uri = $this->uri . $htmlepisode->find('#telecharger',0)->getAttribute('href');
$item->thumbnailUri = $htmlepisode->find('#bigcover', 0)->find('img',0)->getAttribute('src');
$item['id'] = $episode->find('a', 0)->getAttribute('href');
$item['uri'] = $this->uri . $htmlepisode->find('#telecharger',0)->getAttribute('href');
$this->items[] = $item;
}
}
}
public function getName(){
return (!empty($this->request) ? $this->request .' - ' : '') . $this->name;
}
public function getURI(){
return $this->uri;
return $this->getInput('q').' : '.$this->name;
}
public function getCacheDuration(){

61
bridges/CryptomeBridge.php
View File

@ -1,59 +1,40 @@
<?php
class CryptomeBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "BoboTiG";
public $name = "Cryptome";
public $uri = "https://cryptome.org/";
public $description = "Returns the N most recent documents.";
$this->maintainer = "BoboTiG";
$this->name = "Cryptome";
$this->uri = "http://cryptome.org/";
$this->description = "Returns the N most recent documents.";
$this->update = "";
public $parameters = array( array(
'n'=>array(
'name'=>'number of elements',
'type'=>'number',
'defaultValue'=>20,
'exampleValue'=>10
)
));
$this->parameters[] =
'[
{
"name" : "number of elements",
"identifier" : "n",
"type" : "number",
"exampleValue" : "10"
}
]';
}
public function collectData(array $param){
$html = '';
$num = 20;
$link = 'http://cryptome.org/';
// If you want HTTPS access instead, uncomment the following line:
//$link = 'https://secure.netsolhost.com/cryptome.org/';
$html = $this->file_get_html($link) or $this->returnError('Could not request Cryptome.', 404);
if (!empty($param['n'])) { /* number of documents */
$num = min(max(1, $param['n']+0), $num);
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Could not request Cryptome.');
if (!empty($this->getInput('n'))) { /* number of documents */
$num = min($this->getInput('n'), 20);
}
foreach($html->find('pre') as $element) {
for ( $i = 0; $i < $num; ++$i ) {
$item = new \Item();
$item->uri = $link.substr($element->find('a', $i)->href, 20);
$item->title = substr($element->find('b', $i)->plaintext, 22);
$item->content = preg_replace('#http://cryptome.org/#', $link, $element->find('b', $i)->innertext);
$item = array();
$item['uri'] = $this->uri.substr($element->find('a', $i)->href, 20);
$item['title'] = substr($element->find('b', $i)->plaintext, 22);
$item['content'] = preg_replace('#http://cryptome.org/#', $this->uri, $element->find('b', $i)->innertext);
$this->items[] = $item;
}
break;
}
}
public function getName(){
return 'Cryptome';
}
public function getURI(){
return 'https://secure.netsolhost.com/cryptome.org/';
}
public function getCacheDuration(){
return 21600; // 6 hours
}

192
bridges/DailymotionBridge.php
View File

@ -1,107 +1,117 @@
<?php
class DailymotionBridge extends BridgeAbstract{
private $request;
public $maintainer = "mitsukarenai";
public $name = "Dailymotion Bridge";
public $uri = "https://www.dailymotion.com/";
public $description = "Returns the 5 newest videos by username/playlist or search";
public function loadMetadatas() {
public $parameters = array (
'By username' => array(
'u'=>array(
'name'=>'username',
'required'=>true
)
),
$this->maintainer = "mitsukarenai";
$this->name = "Dailymotion Bridge";
$this->uri = "https://www.dailymotion.com/";
$this->description = "Returns the 5 newest videos by username/playlist or search";
$this->update = "2014-11-18";
'By playlist id' => array(
'p'=>array(
'name'=>'playlist id',
'required'=>true
)
),
$this->parameters["By username"] =
'[
{
"name" : "username",
"identifier" : "u"
}
]';
'From search results' => array(
's'=>array(
'name'=>'Search keyword',
'required'=>true
),
'pa'=>array(
'name'=>'Page',
'type'=>'number'
)
)
);
$this->parameters["By playlist id"] =
'[
{
"name" : "playlist id",
"identifier" : "p",
"type" : "number"
}
]';
function getMetadata($id) {
$metadata=array();
$html2 = $this->getSimpleHTMLDOM($this->uri.'video/'.$id);
if(!$html2){
return $metadata;
}
$this->parameters["From search results"] =
'[
{
"name" : "Search keyword",
"identifier" : "s"
},
{
"name" : "Page",
"identifier" : "pa",
"type" : "number"
}
]';
}
$metadata['title'] = $html2->find('meta[property=og:title]', 0)->getAttribute('content');
$metadata['timestamp'] = strtotime($html2->find('meta[property=video:release_date]', 0)->getAttribute('content') );
$metadata['thumbnailUri'] = $html2->find('meta[property=og:image]', 0)->getAttribute('content');
$metadata['uri'] = $html2->find('meta[property=og:url]', 0)->getAttribute('content');
return $metadata;
}
public function collectData(){
$html = '';
$limit = 5;
$count = 0;
public function collectData(array $param){
$html = $this->getSimpleHTMLDOM($this->getURI())
or $this->returnServerError('Could not request Dailymotion.');
function getMetadata($id) {
$metadata=array();
$html2 = $this->file_get_html('http://www.dailymotion.com/video/'.$id) or $this->returnError('Could not request Dailymotion.', 404);
$metadata['title'] = $html2->find('meta[property=og:title]', 0)->getAttribute('content');
$metadata['timestamp'] = strtotime($html2->find('meta[property=video:release_date]', 0)->getAttribute('content') );
$metadata['thumbnailUri'] = $html2->find('meta[property=og:image]', 0)->getAttribute('content');
$metadata['uri'] = $html2->find('meta[property=og:url]', 0)->getAttribute('content');
foreach($html->find('div.media a.preview_link') as $element) {
if($count < $limit) {
$item = array();
$item['id'] = str_replace('/video/', '', strtok($element->href, '_'));
$metadata = $this->getMetadata($item['id']);
if(empty($metadata)){
continue;
}
$item['uri'] = $metadata['uri'];
$item['title'] = $metadata['title'];
$item['timestamp'] = $metadata['timestamp'];
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $metadata['thumbnailUri'] . '" /></a><br><a href="' . $item['uri'] . '">' . $item['title'] . '</a>';
$this->items[] = $item;
$count++;
}
}
}
return $metadata;
}
public function getName(){
switch($this->queriedContext){
case 'By username':
$specific=$this->getInput('u');
break;
case 'By playlist id':
$specific=strtok($this->getInput('p'), '_');
break;
case 'From search results':
$specific=$this->getInput('s');
break;
}
return $specific.' : Dailymotion Bridge';
}
$html = '';
$limit = 5;
$count = 0;
public function getURI(){
$uri=$this->uri;
switch($this->queriedContext){
case 'By username':
$uri.='user/'
.urlencode($this->getInput('u')).'/1';
break;
case 'By playlist id':
$uri.='playlist/'
.urlencode(strtok($this->getInput('p'), '_'));
break;
case 'From search results':
$uri.='search/'
.urlencode($this->getInput('s'));
if($this->getInput('pa')){
$uri.='/'.$this->getInput('pa');
}
break;
}
return $uri;
}
if (isset($param['u'])) { // user timeline mode
$this->request = $param['u'];
$html = $this->file_get_html('http://www.dailymotion.com/user/'.urlencode($this->request).'/1') or $this->returnError('Could not request Dailymotion.', 404);
}
else if (isset($param['p'])) { // playlist mode
$this->request = strtok($param['p'], '_');
$html = $this->file_get_html('http://www.dailymotion.com/playlist/'.urlencode($this->request).'') or $this->returnError('Could not request Dailymotion.', 404);
}
else if (isset($param['s'])) { // search mode
$this->request = $param['s']; $page = 1; if (isset($param['pa'])) $page = (int)preg_replace("/[^0-9]/",'', $param['pa']);
$html = $this->file_get_html('http://www.dailymotion.com/search/'.urlencode($this->request).'/'.$page.'') or $this->returnError('Could not request Dailymotion.', 404);
}
else {
$this->returnError('You must either specify a Dailymotion username (?u=...) or a playlist id (?p=...) or search (?s=...)', 400);
}
foreach($html->find('div.media a.preview_link') as $element) {
if($count < $limit) {
$item = new \Item();
$item->id = str_replace('/video/', '', strtok($element->href, '_'));
$metadata = getMetadata($item->id);
$item->uri = $metadata['uri'];
$item->thumbnailUri = $metadata['thumbnailUri'];
$item->title = $metadata['title'];
$item->timestamp = $metadata['timestamp'];
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" /></a><br><a href="' . $item->uri . '">' . $item->title . '</a>';
$this->items[] = $item;
$count++;
}
}
}
public function getName(){
return (!empty($this->request) ? $this->request .' - ' : '') .'Dailymotion Bridge';
}
public function getURI(){
return 'https://www.dailymotion.com/';
}
public function getCacheDuration(){
return 3600*3; // 3 hours
}
public function getCacheDuration(){
return 3600*3; // 3 hours
}
}

70
bridges/DanbooruBridge.php
View File

@ -1,58 +1,38 @@
<?php
class DanbooruBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "mitsukarenai";
public $name = "Danbooru";
public $uri = "http://donmai.us/";
public $description = "Returns images from given page";
$this->maintainer = "mitsukarenai";
$this->name = "Danbooru";
$this->uri = "http://donmai.us/";
$this->description = "Returns images from given page";
$this->update = "2014-05-25";
public $parameters = array( array(
'p'=>array(
'name'=>'page',
'type'=>'number'
),
't'=>array('name'=>'tags')
));
$this->parameters[] =
'[
{
"name" : "page",
"identifier" : "p"
},
{
"name" : "tags",
"identifier" : "t"
}
]';
public function collectData(){
$page = $this->getInput('p')?$this->getInput('p'):1;
$tags = urlencode($this->getInput('t'));
}
public function collectData(array $param){
$page = 1;$tags='';
if (isset($param['p'])) {
$page = (int)preg_replace("/[^0-9]/",'', $param['p']);
}
if (isset($param['t'])) {
$tags = urlencode($param['t']);
}
$html = $this->file_get_html("http://donmai.us/posts?&page=$page&tags=$tags") or $this->returnError('Could not request Danbooru.', 404);
$html = $this->getSimpleHTMLDOM($this->uri."posts?&page=$page&tags=$tags")
or $this->returnServerError('Could not request Danbooru.');
foreach($html->find('div[id=posts] article') as $element) {
$item = new \Item();
$item->uri = 'http://donmai.us'.$element->find('a', 0)->href;
$item->postid = (int)preg_replace("/[^0-9]/",'', $element->getAttribute('data-id'));
$item->timestamp = time();
$item->thumbnailUri = 'http://donmai.us'.$element->find('img', 0)->src;
$item->tags = $element->find('img', 0)->getAttribute('alt');
$item->title = 'Danbooru | '.$item->postid;
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" /></a><br>Tags: '.$item->tags;
$this->items[] = $item;
$item = array();
$item['uri'] = $this->uri.$element->find('a', 0)->href;
$item['postid'] = (int)preg_replace("/[^0-9]/",'', $element->getAttribute('data-id'));
$item['timestamp'] = time();
$thumbnailUri = $this->uri.$element->find('img', 0)->src;
$item['tags'] = $element->find('img', 0)->getAttribute('alt');
$item['title'] = 'Danbooru | '.$item['postid'];
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $thumbnailUri . '" /></a><br>Tags: '.$item['tags'];
$this->items[] = $item;
}
}
public function getName(){
return 'Danbooru';
}
public function getURI(){
return 'http://donmai.us/';
}
public function getCacheDuration(){
return 1800; // 30 minutes
}

36
bridges/DansTonChatBridge.php
View File

@ -1,39 +1,25 @@
<?php
class DansTonChatBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "Astalaseven";
public $name = "DansTonChat Bridge";
public $uri = "http://danstonchat.com/";
public $description = "Returns latest quotes from DansTonChat.";
$this->maintainer = "Astalaseven";
$this->name = "DansTonChat Bridge";
$this->uri = "http://danstonchat.com/latest.html";
$this->description = "Returns latest quotes from DansTonChat.";
$this->update = "2014-05-25";
public function collectData(){
}
public function collectData(array $param){
$html = '';
$link = 'http://danstonchat.com/latest.html';
$html = $this->file_get_html($link) or $this->returnError('Could not request DansTonChat.', 404);
$html = $this->getSimpleHTMLDOM($this->uri.'latest.html')
or $this->returnServerError('Could not request DansTonChat.');
foreach($html->find('div.item') as $element) {
$item = new \Item();
$item->uri = $element->find('a', 0)->href;
$item->title = 'DansTonChat '.$element->find('a', 1)->plaintext;
$item->content = $element->find('a', 0)->innertext;
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$item['title'] = 'DansTonChat '.$element->find('a', 1)->plaintext;
$item['content'] = $element->find('a', 0)->innertext;
$this->items[] = $item;
}
}
public function getName(){
return 'DansTonChat';
}
public function getURI(){
return 'http://danstonchat.com';
}
public function getCacheDuration(){
return 21600; // 6 hours
}

172
bridges/DauphineLibereBridge.php
View File

@ -1,130 +1,70 @@
<?php
class DauphineLibereBridge extends BridgeAbstract{
class DauphineLibereBridge extends BridgeAbstract {
public function loadMetadatas() {
public $maintainer = "qwertygc";
public $name = "Dauphine Bridge";
public $uri = "http://www.ledauphine.com/";
public $description = "Returns the newest articles.";
$this->maintainer = "qwertygc";
$this->name = "DauphineLibereBridge Bridge";
$this->uri = "http://www.ledauphine.com/";
$this->description = "Returns the newest articles.";
$this->update = "05/11/2015";
public $parameters = array( array(
'u'=>array(
'name'=>'Catégorie de l\'article',
'type'=>'list',
'values'=>array(
'À la une'=>'',
'France Monde'=>'france-monde',
'Faits Divers'=>'faits-divers',
'Économie et Finance'=>'economie-et-finance',
'Politique'=>'politique',
'Sport'=>'sport',
'Ain'=>'ain',
'Alpes-de-Haute-Provence'=>'haute-provence',
'Hautes-Alpes'=>'hautes-alpes',
'Ardèche'=>'ardeche',
'Drôme'=>'drome',
'Isère Sud'=>'isere-sud',
'Savoie'=>'savoie',
'Haute-Savoie'=>'haute-savoie',
'Vaucluse'=>'vaucluse'
)
)
));
$this->parameters[] =
'[
{
"name" : "Catégorie de l\'article",
"identifier" : "u",
"type" : "list",
"values" : [
{
"name" : "À la une",
"value" : ""
},
{
"name" : "France Monde",
"value" : "france-monde"
},
{
"name" : "Faits Divers",
"value" : "faits-divers"
},
{
"name" : "Économie et Finance",
"value" : "economie-et-finance"
},
{
"name" : "Politique",
"value" : "politique"
},
{
"name" : "Sport",
"value" : "sport"
},
{
"name" : "Ain",
"value" : "ain"
},
{
"name" : "Alpes-de-Haute-Provence",
"value" : "haute-provence"
},
{
"name" : "Hautes-Alpes",
"value" : "hautes-alpes"
},
{
"name" : "Ardèche",
"value" : "ardeche"
},
{
"name" : "Drôme",
"value" : "drome"
},
{
"name" : "Isère Sud",
"value" : "isere-sud"
},
{
"name" : "Savoie",
"value" : "savoie"
},
{
"name" : "Haute-Savoie",
"value" : "haute-savoie"
},
{
"name" : "Vaucluse",
"value" : "vaucluse"
}
]
}
]';
}
public function collectData(array $param){
function ExtractContent($url) {
$html2 = $this->file_get_html($url);
private function ExtractContent($url, $context) {
$html2 = $this->getSimpleHTMLDOM($url);
$text = $html2->find('div.column', 0)->innertext;
$text = preg_replace('@<script[^>]*?>.*?</script>@si', '', $text);
return $text;
}
if (isset($param['u'])) { /* user timeline mode */
$this->request = $param['u'];
$html = $this->file_get_html('http://www.ledauphine.com/'.$this->request.'/rss') or $this->returnError('Could not request DauphineLibere.', 404);
}
else {
$html = $this->file_get_html('http://www.ledauphine.com/rss') or $this->returnError('Could not request DauphineLibere.', 404);
}
public function collectData(){
$context = stream_context_create($opts);
if (empty($this->getInput('u'))) {
$html = $this->getSimpleHTMLDOM($this->uri.$this->getInput('u').'/rss')
or $this->returnServerError('Could not request DauphineLibere.');
} else {
$html = $this->getSimpleHTMLDOM($this->uri.'rss')
or $this->returnServerError('Could not request DauphineLibere.');
}
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 10) {
$item = new \Item();
$item->title = $element->find('title', 0)->innertext;
$item->uri = $element->find('guid', 0)->plaintext;
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = ExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
if($limit < 10) {
$item = array();
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = $element->find('guid', 0)->plaintext;
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $this->ExtractContent($item['uri'], $context);
$this->items[] = $item;
$limit++;
}
}
}
}
public function getName(){
return 'Dauphine Bridge';
}
public function getURI(){
return 'http://ledauphine.com/';
}
public function getCacheDuration(){
return 3600*2; // 2 hours
// return 0; // 2 hours
}
public function getCacheDuration(){
return 3600*2; // 2 hours
}
}
?>

110
bridges/DemoBridge.php
View File

@ -1,83 +1,53 @@
<?php
class DemoBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "teromene";
public $name = "DemoBridge";
public $uri = "http://github.com/rss-bridge/rss-bridge";
public $description = "Bridge used for demos";
$this->maintainer = "teromene";
$this->name = "DemoBridge";
$this->uri = "http://github.com/sebsauvage/rss-bridge";
$this->description = "Bridge used for demos";
$this->update = "2015-11-03";
public $parameters = array(
'testCheckbox' => array(
'testCheckbox'=>array(
'type'=>'checkbox',
'name'=>'test des checkbox'
)
),
$this->parameters['testCheckbox'] =
'[
{
"type" : "checkbox",
"identifier" : "testCheckbox",
"name" : "test des checkbox"
}
'testList' => array(
'testList'=>array(
'type'=>'list',
'name'=>'test des listes',
'values'=>array(
'Test'=>'test',
'Test 2'=>'test2'
)
)
),
]';
'testNumber' => array(
'testNumber'=>array(
'type'=>'number',
'name'=>'test des numéros',
'exampleValue'=>'1515632'
)
)
);
$this->parameters['testList'] =
'[
{
"type" : "list",
"identifier" : "testList",
"name" : "test des listes",
"values" : [
{
"name" : "Test",
"value" : "test"
},
{
"name" : "Test 2",
"value" : "test2"
}
]
}
]';
$this->parameters['testNumber'] =
'[
{
"type" : "number",
"identifier" : "testNumber",
"name" : "test des numéros",
"exampleValue" : "1515632"
public function collectData(){
}
$item = array();
$item['author'] = "Me!";
$item['title'] = "Test";
$item['content'] = "Awesome content !";
$item['id'] = "Lalala";
$item['uri'] = "http://example.com/test";
]';
}
public function collectData(array $param){
$item = new \Item();
$item->name = "TestElement";
$item->title = "Test";
$item->content = "Awesome content !";
$item->id = "Lalala";
$item->uri = "http://test.test/test";
$item->enclosures[] = "http://www.ardmediathek.de/ard/servlet/image/00/32/68/18/38/1135274624/16x9/960";
$this->items[] = $item;
$this->items[] = $item;
}
public function getName() {
return "DemoBridge";
}
public function getURI() {
return "http://github.com/sebsauvage/rss-bridge";
}
public function getCacheDuration(){
return 00; // 1 hour
}
public function getCacheDuration(){
return 00; // 1 hour
}
}

112
bridges/DeveloppezDotComBridge.php
View File

@ -1,75 +1,63 @@
<?php
class DeveloppezDotComBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "polopollo";
$this->name = "Developpez.com Actus (FR)";
$this->uri = "http://www.developpez.com/";
$this->description = "Returns the 15 newest posts from DeveloppezDotCom (full text).";
$this->update = "2014-07-14";
public $maintainer = "polopollo";
public $name = "Developpez.com Actus (FR)";
public $uri = "http://www.developpez.com/";
public $description = "Returns the 15 newest posts from DeveloppezDotCom (full text).";
private function DeveloppezDotComStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
public function collectData(array $param){
// F***ing quotes from Microsoft Word badly encoded, here was the trick:
// http://stackoverflow.com/questions/1262038/how-to-replace-microsoft-encoded-quotes-in-php
private function convert_smart_quotes($string)
{
$search = array(chr(145),
chr(146),
chr(147),
chr(148),
chr(151));
function DeveloppezDotComStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
$replace = array("'",
"'",
'"',
'"',
'-');
function convert_smart_quotes($string)//F***ing quotes from Microsoft Word badly encoded, here was the trick: http://stackoverflow.com/questions/1262038/how-to-replace-microsoft-encoded-quotes-in-php
{
$search = array(chr(145),
chr(146),
chr(147),
chr(148),
chr(151));
return str_replace($search, $replace, $string);
}
$replace = array("'",
"'",
'"',
'"',
'-');
private function DeveloppezDotComExtractContent($url) {
$articleHTMLContent = $this->getSimpleHTMLDOM($url);
$text = $this->convert_smart_quotes($articleHTMLContent->find('div.content', 0)->innertext);
$text = utf8_encode($text);
return trim($text);
}
return str_replace($search, $replace, $string);
}
public function collectData(){
$rssFeed = $this->getSimpleHTMLDOM($this->uri.'index/rss')
or $this->returnServerError('Could not request '.$this->uri.'index/rss');
$limit = 0;
function DeveloppezDotComExtractContent($url) {
$articleHTMLContent = $this->file_get_html($url);
$text = convert_smart_quotes($articleHTMLContent->find('div.content', 0)->innertext);
$text = utf8_encode($text);
return trim($text);
}
foreach($rssFeed->find('item') as $element) {
if($limit < 10) {
$item = array();
$item['title'] = $this->DeveloppezDotComStripCDATA($element->find('title', 0)->innertext);
$item['uri'] = $this->DeveloppezDotComStripCDATA($element->find('guid', 0)->plaintext);
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$content = $this->DeveloppezDotComExtractContent($item['uri']);
$item['content'] = strlen($content) ? $content : $element->description; //In case of it is a tutorial, we just keep the original description
$this->items[] = $item;
$limit++;
}
}
}
$rssFeed = $this->file_get_html('http://www.developpez.com/index/rss') or $this->returnError('Could not request http://www.developpez.com/index/rss', 404);
$limit = 0;
foreach($rssFeed->find('item') as $element) {
if($limit < 10) {
$item = new \Item();
$item->title = DeveloppezDotComStripCDATA($element->find('title', 0)->innertext);
$item->uri = DeveloppezDotComStripCDATA($element->find('guid', 0)->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$content = DeveloppezDotComExtractContent($item->uri);
$item->content = strlen($content) ? $content : $element->description;//In case of it is a tutorial, we just keep the original description
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'DeveloppezDotCom';
}
public function getURI(){
return 'http://www.developpez.com/';
}
public function getCacheDuration(){
return 1800; // 30min
}
public function getCacheDuration(){
return 1800; // 30min
}
}

42
bridges/DilbertBridge.php
View File

@ -1,19 +1,14 @@
<?php
class DilbertBridge extends BridgeAbstract {
public function loadMetadatas() {
public $maintainer = 'kranack';
public $name = 'Dilbert Daily Strip';
public $uri = 'http://dilbert.com';
public $description = 'The Unofficial Dilbert Daily Comic Strip';
$this->maintainer = 'kranack';
$this->name = $this->getName();
$this->uri = $this->getURI();
$this->description = $this->getDescription();
$this->update = "14/05/2016";
public function collectData(){
}
public function collectData(array $param) {
$html = $this->file_get_html($this->getURI()) or $this->returnError('Could not request Dilbert: '.$this->getURI(), 500);
$html = $this->getSimpleHTMLDOM($this->getURI()) or $this->returnServerError('Could not request Dilbert: '.$this->getURI());
foreach ($html->find('section.comic-item') as $element) {
@ -27,29 +22,16 @@ class DilbertBridge extends BridgeAbstract {
$title = 'Dilbert Comic Strip on '.$date;
$date = strtotime($date);
$item = new \Item();
$item->uri = $url;
$item->thumbnailUri = $comic;
$item->title = $title;
$item->author = 'Scott Adams';
$item->timestamp = $date;
$item->content = '<img src="'.$comic.'" alt="'.$img->alt.'" />';
$item = array();
$item['uri'] = $url;
$item['title'] = $title;
$item['author'] = 'Scott Adams';
$item['timestamp'] = $date;
$item['content'] = '<img src="'.$comic.'" alt="'.$img->alt.'" />';
$this->items[] = $item;
}
}
public function getName() {
return 'Dilbert Daily Strip';
}
public function getURI() {
return 'http://dilbert.com';
}
public function getDescription() {
return 'The Unofficial Dilbert Daily Comic Strip';
}
public function getCacheDuration() {
return 21600; // 6 hours
}

71
bridges/DollbooruBridge.php
View File

@ -1,61 +1,40 @@
<?php
class DollbooruBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "mitsukarenai";
$this->name = "Dollbooru";
$this->uri = "http://dollbooru.org/";
$this->description = "Returns images from given page";
$this->update = "2015-01-20";
public $maintainer = "mitsukarenai";
public $name = "Dollbooru";
public $uri = "http://dollbooru.org/";
public $description = "Returns images from given page";
$this->parameters[] =
'[
{
"name" : "page",
"type" : "number",
"identifier" : "p"
},
{
"name" : "tags",
"identifier" : "t"
}
]';
}
public $parameters = array( array(
'p'=>array(
'name'=>'page',
'type'=>'number'
),
't'=>array('name'=>'tags')
));
public function collectData(array $param){
$page = 0;$tags='';
if (isset($param['p'])) {
$page = (int)preg_replace("/[^0-9]/",'', $param['p']);
}
if (isset($param['t'])) {
$tags = urlencode($param['t']);
}
$html = $this->file_get_html("http://dollbooru.org/post/list/$tags/$page") or $this->returnError('Could not request Dollbooru.', 404);
public function collectData(){
$page=$this->getInput('p');
$tags = urlencode($this->getInput('t'));
$html = $this->getSimpleHTMLDOM($this->uri."post/list/$tags/$page")
or $this->returnServerError('Could not request Dollbooru.');
foreach($html->find('div[class=shm-image-list] a') as $element) {
$item = new \Item();
$item->uri = 'http://dollbooru.org'.$element->href;
$item->postid = (int)preg_replace("/[^0-9]/",'', $element->getAttribute('data-post-id'));
$item->timestamp = time();
$item->thumbnailUri = 'http://dollbooru.org'.$element->find('img', 0)->src;
$item->tags = $element->getAttribute('data-tags');
$item->title = 'Dollbooru | '.$item->postid;
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" /></a><br>Tags: '.$item->tags;
$this->items[] = $item;
$item = array();
$item['uri'] = $this->uri.$element->href;
$item['postid'] = (int)preg_replace("/[^0-9]/",'', $element->getAttribute('data-post-id'));
$item['timestamp'] = time();
$thumbnailUri = $this->uri.$element->find('img', 0)->src;
$item['tags'] = $element->getAttribute('data-tags');
$item['title'] = 'Dollbooru | '.$item['postid'];
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $thumbnailUri . '" /></a><br>Tags: '.$item['tags'];
$this->items[] = $item;
}
}
public function getName(){
return 'Dollbooru';
}
public function getURI(){
return 'http://dollbooru.org/';
}
public function getCacheDuration(){
return 1800; // 30 minutes
}

48
bridges/DuckDuckGoBridge.php
View File

@ -1,46 +1,30 @@
<?php
class DuckDuckGoBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "Astalaseven";
public $name = "DuckDuckGo";
public $uri = "https://duckduckgo.com/";
public $description = "Returns most recent results from DuckDuckGo.";
$this->maintainer = "Astalaseven";
$this->name = "DuckDuckGo";
$this->uri = "https://duckduckgo.com/";
$this->description = "Returns most recent results from DuckDuckGo.";
$this->update = "2014-05-25";
public $parameters = array( array(
'u'=>array(
'name'=>'keyword',
'required'=>true)
));
$this->parameters[] =
'[
{
"name" : "keyword",
"identifier" : "u"
}
]';
}
public function collectData(array $param){
$html = '';
$link = 'http://duckduckgo.com/html/?q='.$param[u].'+sort:date';
$html = $this->file_get_html($link) or $this->returnError('Could not request DuckDuckGo.', 404);
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri.'html/?q='.$this->getInput('u').'+sort:date')
or $this->returnServerError('Could not request DuckDuckGo.');
foreach($html->find('div.results_links') as $element) {
$item = new \Item();
$item->uri = $element->find('a', 0)->href;
$item->title = $element->find('a', 1)->innertext;
$item->content = $element->find('div.snippet', 0)->plaintext;
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$item['title'] = $element->find('a', 1)->innertext;
$item['content'] = $element->find('div.snippet', 0)->plaintext;
$this->items[] = $item;
}
}
public function getName(){
return 'DuckDuckGo';
}
public function getURI(){
return 'https://duckduckgo.com';
}
public function getCacheDuration(){
return 21600; // 6 hours
}

65
bridges/EZTVBridge.php
View File

@ -1,26 +1,20 @@
<?php
class EZTVBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "alexAubin";
public $name = "EZTV";
public $uri = "https://eztv.ch/";
public $description = "Returns list of *recent* torrents for a specific show on EZTV. Get showID from URLs in https://eztv.ch/shows/showID/show-full-name.";
$this->maintainer = "alexAubin";
$this->name = "EZTV";
$this->uri = "https://eztv.ch/";
$this->description = "Returns list of *recent* torrents for a specific show on EZTV. Get showID from URLs in https://eztv.ch/shows/showID/show-full-name.";
$this->update = "2014-01-20";
public $parameters = array( array(
'i'=>array(
'name'=>'Show ids',
'exampleValue'=>'showID1,showID2,…',
'required'=>true
)
));
$this->parameters[] =
'[
{
"name" : "Show ids",
"identifier" : "i",
"exampleValue" : "showID1,showID2,..."
}
]';
}
public function collectData(array $param){
public function collectData(){
// Make timestamp from relative released time in table
function makeTimestamp($relativeReleaseTime){
@ -35,16 +29,13 @@ class EZTVBridge extends BridgeAbstract{
return mktime(date('h')-$relativeHours,0,0,date('m'),date('d')-$relativeDays,date('Y'));
}
// Check for ID provided
if (!isset($param['i']))
$this->returnError('You must provide a list of ID (?i=showID1,showID2,...)', 400);
// Loop on show ids
$showList = explode(",",$param['i']);
$showList = explode(",",$this->getInput('i'));
foreach($showList as $showID){
// Get show page
$html = $this->file_get_html('https://eztv.ch/shows/'.rawurlencode($showID).'/') or $this->returnError('Could not request EZTV for id "'.$showID.'"', 404);
$html = $this->getSimpleHTMLDOM($this->uri.'shows/'.rawurlencode($showID).'/')
or $this->returnServerError('Could not request EZTV for id "'.$showID.'"');
// Loop on each element that look like an episode entry...
foreach($html->find('.forum_header_border') as $element) {
@ -61,27 +52,15 @@ class EZTVBridge extends BridgeAbstract{
if ($released->plaintext == '&gt;1 week') continue;
// Fill item
$item = new \Item();
$item->uri = 'https://eztv.ch/'.$epinfo->href;
$item->id = $item->uri;
$item->timestamp = makeTimestamp($released->plaintext);
$item->title = $epinfo->plaintext;
$item->content = $epinfo->alt;
if(!empty($item->title))
$item = array();
$item['uri'] = $this->uri.$epinfo->href;
$item['id'] = $item['uri'];
$item['timestamp'] = makeTimestamp($released->plaintext);
$item['title'] = $epinfo->plaintext;
$item['content'] = $epinfo->alt;
if(isset($item['title']))
$this->items[] = $item;
}
}
}
public function getName(){
return 'EZTV';
}
public function getURI(){
return 'https://eztv.ch/';
}
public function getCacheDuration(){
return 3600; // 1 hour
}
}

42
bridges/EliteDangerousGalnetBridge.php
View File

@ -1,53 +1,39 @@
<?php
class EliteDangerousGalnetBridge extends BridgeAbstract
{
public function loadMetadatas()
{
$this->maintainer = "corenting";
$this->name = "Elite: Dangerous Galnet";
$this->uri = "https://community.elitedangerous.com/galnet";
$this->description = "Returns the latest page of news from Galnet";
$this->update = "2016-04-06";
}
public $maintainer = "corenting";
public $name = "Elite: Dangerous Galnet";
public $uri = "https://community.elitedangerous.com/galnet/";
public $description = "Returns the latest page of news from Galnet";
public function collectData(array $param)
public function collectData()
{
$html = $this->file_get_html('https://community.elitedangerous.com/galnet') or $this->returnError('Error while downloading the website content', 404);
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Error while downloading the website content');
foreach($html->find('div.article') as $element) {
$item = new Item();
$item = array();
$uri = $element->find('h3 a', 0)->href;
$uri = 'https://community.elitedangerous.com' . $uri;
$item->uri = $uri;
$uri = $this->uri . substr($uri,strlen('/galnet/'));
$item['uri'] = $uri;
$title = $element->find('h3 a', 0)->plaintext;
$item->title = substr($title, 1); //remove the space between icon and title
$item['title'] = substr($title, 1); //remove the space between icon and title
$content = $element->find('p', -1)->innertext;
$item->content = $content;
$item['content'] = $content;
$date = $element->find('p.small', 0)->innertext;
$article_year = substr($date, -4) - 1286; //Convert E:D date to actual date
$date = substr($date, 0, -4) . $article_year;
$item->timestamp = strtotime($date);
$item['timestamp'] = strtotime($date);
$this->items[] = $item;
}
}
public function getName()
{
return 'Elite: Dangerous Galnet';
}
public function getURI()
{
return 'https://community.elitedangerous.com/galnet';
}
public function getCacheDuration()
{
return 3600 * 2; // 2 hours
}
}
}

112
bridges/ElsevierBridge.php
View File

@ -1,55 +1,77 @@
<?php
/**
* ElsevierBridge
*
* @name Elsevier Bridge
* @description Returns the recent articles published in Elsevier journals
*/
class ElsevierBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = 'Pierre Mazière';
public $name = 'Elsevier journals recent articles';
public $uri = 'http://www.journals.elsevier.com/';
public $description = 'Returns the recent articles published in Elsevier journals';
$this->maintainer = 'Pierre Mazière';
$this->name = 'Elsevier journals recent articles';
$this->uri = 'http://www.journals.elsevier.com';
$this->description = 'Returns the recent articles published in Elsevier journals';
$this->update = '2016-06-26';
public $parameters = array( array(
'j'=>array(
'name'=>'Journal name',
'required'=>true,
'exampleValue'=>'academic-pediactrics',
'title'=>'Insert html-part of your journal'
)
));
$this->parameters=
'[
{
"name" : "Journal name",
"identifier" : "j"
}
]';
}
// Extracts the list of names from an article as string
private function ExtractArticleName ($article){
$names = $article->find('small', 0);
if($names)
return trim($names->plaintext);
return '';
}
public function collectData(array $param){
$uri = 'http://www.journals.elsevier.com/'.$param['j'].'/recent-articles/';
$html = file_get_html($uri)
or $this->returnError('No results for Elsevier journal '.$param['j'], 404);
// Extracts the timestamp from an article
private function ExtractArticleTimestamp ($article){
$time = $article->find('.article-info', 0);
if($time){
$timestring = trim($time->plaintext);
/*
The format depends on the age of an article:
- Available online 29 July 2016
- July 2016
- May–June 2016
*/
if(preg_match('/\S*(\d+\s\S+\s\d{4})/ims', $timestring, $matches)){
return strtotime($matches[0]);
} elseif (preg_match('/[A-Za-z]+\-([A-Za-z]+\s\d{4})/ims', $timestring, $matches)){
return strtotime($matches[0]);
} elseif (preg_match('/([A-Za-z]+\s\d{4})/ims', $timestring, $matches)){
return strtotime($matches[0]);
} else {
return 0;
}
}
return 0;
}
foreach($html->find('.pod-listing') as $article){
// Extracts the content from an article
private function ExtractArticleContent ($article){
$content = $article->find('.article-content', 0);
if($content){
return trim($content->plaintext);
}
return '';
}
$item = new \Item();
$item->uri=$article->find('.pod-listing-header>a',0)->getAttribute('href').'?np=y';
$item->title=$article->find('.pod-listing-header>a',0)->plaintext;
$item->name=trim($article->find('small',0)->plaintext);
$item->timestamp=strtotime($article->find('.article-info',0)->plaintext);
$item->content=trim($article->find('.article-content',0)->plaintext);
public function collectData(){
$uri = $this->uri . $this->getInput('j') . '/recent-articles/';
$html = $this->getSimpleHTMLDOM($uri) or $this->returnServerError('No results for Elsevier journal '.$this->getInput('j'));
$this->items[]=$item;
}
}
foreach($html->find('.pod-listing') as $article){
$item = array();
$item['uri'] = $article->find('.pod-listing-header>a',0)->getAttribute('href').'?np=y';
$item['title'] = $article->find('.pod-listing-header>a',0)->plaintext;
$item['author'] = $this->ExtractArticleName($article);
$item['timestamp'] = $this->ExtractArticleTimestamp($article);
$item['content'] = $this->ExtractArticleContent($article);
$this->items[] = $item;
}
}
public function getName(){
return 'Elsevier journals recent articles';
}
public function getURI(){
return 'http://www.journals.elsevier.com';
}
public function getCacheDuration(){
return 43200; // 12h
}
public function getCacheDuration(){
return 43200; // 12h
}
}
?>

49
bridges/EstCeQuonMetEnProdBridge.php
View File

@ -1,15 +1,12 @@
<?php
class EstCeQuonMetEnProdBridge extends BridgeAbstract {
public function loadMetadatas() {
$this->maintainer = 'ORelio';
$this->name = $this->getName();
$this->uri = $this->getURI();
$this->description = $this->getDescription();
$this->update = "02/07/2016";
}
public $maintainer = 'ORelio';
public $name = 'Est-ce qu\'on met en prod aujourd\'hui ?';
public $uri = 'https://www.estcequonmetenprodaujourdhui.info/';
public $description = 'Should we put a website in production today? (French)';
public function collectData(array $param) {
public function collectData(){
function ExtractFromDelimiters($string, $start, $end) {
if (strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
@ -18,39 +15,19 @@ class EstCeQuonMetEnProdBridge extends BridgeAbstract {
} return false;
}
$html = $this->file_get_html($this->getURI()) or $this->returnError('Could not request EstCeQuonMetEnProd: '.$this->getURI(), 500);
$html = $this->getSimpleHTMLDOM($this->getURI()) or $this->returnServerError('Could not request EstCeQuonMetEnProd: '.$this->getURI());
$img = $html->find('img', 0);
if (is_object($img)) {
$img = $img->src;
if ($img[0] == '/')
$img = substr($this->getURI(), 0, strlen($this->getURI()) - 1).$img;
}
$item = new \Item();
$item->uri = $this->getURI().'#'.date('Y-m-d');
$item->thumbnailUri = $img;
$item->title = $this->getName();
$item->author = 'Nicolas Hoffmann';
$item->timestamp = strtotime('today midnight');
$item->content = str_replace('src="/', 'src="'.$this->getURI(), trim(ExtractFromDelimiters($html->outertext, '<body role="document">', '<br /><br />')));
$item = array();
$item['uri'] = $this->getURI().'#'.date('Y-m-d');
$item['title'] = $this->getName();
$item['author'] = 'Nicolas Hoffmann';
$item['timestamp'] = strtotime('today midnight');
$item['content'] = str_replace('src="/', 'src="'.$this->getURI(), trim(ExtractFromDelimiters($html->outertext, '<body role="document">', '<br /><br />')));
$this->items[] = $item;
}
public function getName() {
return 'Est-ce qu\'on met en prod aujourd\'hui ?';
}
public function getURI() {
return 'https://www.estcequonmetenprodaujourdhui.info/';
}
public function getDescription() {
return 'Should we put a website in production today? (French)';
}
public function getCacheDuration() {
return 21600; // 6 hours
}
}
?>
?>

87
bridges/FacebookBridge.php
View File

@ -1,25 +1,21 @@
<?php
class FacebookBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "teromene";
public $name = "Facebook";
public $uri = "https://www.facebook.com/";
public $description = "Input a page title or a profile log. For a profile log, please insert the parameter as follow : myExamplePage/132621766841117";
$this->maintainer = "teromene";
$this->name = "Facebook";
$this->uri = "http://www.facebook.com/";
$this->description = "Input a page title or a profile log. For a profile log, please insert the parameter as follow : myExamplePage/132621766841117";
$this->update = "31/03/2016";
public $parameters =array( array(
'u'=>array(
'name'=>'Username',
'required'=>true
)
));
$this->parameters[] =
'[
{
"name" : "Username",
"identifier" : "u",
"required" : "required"
}
]';
}
private $authorName='';
public function collectData(array $param) {
public function collectData(){
//Extract a string using start and end delimiters
function ExtractFromDelimiters($string, $start, $end) {
@ -35,7 +31,7 @@ class FacebookBridge extends BridgeAbstract{
if (is_array($matches) && count($matches) > 1) {
$link = $matches[1];
if (strpos($link, '/') === 0)
$link = 'https://www.facebook.com'.$link.'"';
$link = $this->uri.$link.'"';
if (strpos($link, 'facebook.com/l.php?u=') !== false)
$link = urldecode(ExtractFromDelimiters($link, 'facebook.com/l.php?u=', '&'));
return ' href="'.$link.'"';
@ -96,8 +92,8 @@ class FacebookBridge extends BridgeAbstract{
),
);
$context = stream_context_create($http_options);
$html = file_get_contents($captcha_action, false, $context);
if ($html === FALSE) { $this->returnError('Failed to submit captcha response back to Facebook', 500); }
$html = $this->getContents($captcha_action, false, $context);
if ($html === FALSE) { $this->returnServerError('Failed to submit captcha response back to Facebook'); }
unset($_SESSION['captcha_fields']);
$html = str_get_html($html);
}
@ -107,14 +103,12 @@ class FacebookBridge extends BridgeAbstract{
//Retrieve page contents
if (is_null($html)) {
if (isset($param['u'])) {
if (!strpos($param['u'], "/")) {
$html = $this->file_get_html('https://www.facebook.com/'.urlencode($param['u']).'?_fb_noscript=1') or $this->returnError('No results for this query.', 404);
} else {
$html = $this->file_get_html('https://www.facebook.com/pages/'.$param['u'].'?_fb_noscript=1') or $this->returnError('No results for this query.', 404);
}
if (!strpos($this->getInput('u'), "/")) {
$html = $this->getSimpleHTMLDOM($this->uri.urlencode($this->getInput('u')).'?_fb_noscript=1')
or $this->returnServerError('No results for this query.');
} else {
$this->returnError('You must specify a Facebook username.', 400);
$html = $this->getSimpleHTMLDOM($this->uri.'pages/'.$this->getInput('u').'?_fb_noscript=1')
or $this->returnServerError('No results for this query.');
}
}
@ -129,10 +123,10 @@ class FacebookBridge extends BridgeAbstract{
foreach ($captcha->find('input, button') as $input)
$captcha_fields[$input->name] = $input->value;
$_SESSION['captcha_fields'] = $captcha_fields;
$_SESSION['captcha_action'] = 'https://www.facebook.com'.$captcha->find('form', 0)->action;
$_SESSION['captcha_action'] = $this->uri.$captcha->find('form', 0)->action;
//Show captcha filling form to the viewer, proxying the captcha image
$img = base64_encode(file_get_contents($captcha->find('img', 0)->src));
$img = base64_encode($this->getContents($captcha->find('img', 0)->src));
header('HTTP/1.1 500 '.Http::getMessageForCode(500));
header('Content-Type: text/html');
die('<form method="post" action="?'.$_SERVER['QUERY_STRING'].'">'
@ -151,12 +145,12 @@ class FacebookBridge extends BridgeAbstract{
if(isset($element)) {
$author = str_replace(' | Facebook', '', $html->find('title#pageTitle', 0)->innertext);
$profilePic = 'https://graph.facebook.com/'.$param['u'].'/picture?width=200&amp;height=200';
$this->name = $author;
$profilePic = 'https://graph.facebook.com/'.$this->getInput('u').'/picture?width=200&amp;height=200';
$this->authorName = $author;
foreach($element->children() as $post) {
$item = new \Item();
$item = array();
if (count($post->find('abbr')) > 0) {
@ -197,37 +191,20 @@ class FacebookBridge extends BridgeAbstract{
if (strlen($title) > 64)
$title = substr($title, 0, strpos(wordwrap($title, 64), "\n")).'...';
//Use first image as thumbnail if available, or profile pic fallback
$thumbnail = $post->find('img', 1);
if (is_object($thumbnail))
$thumbnail = $thumbnail->src;
else $thumbnail = $profilePic;
//Build and add final item
$item->uri = 'https://facebook.com'.$post->find('abbr')[0]->parent()->getAttribute('href');
$item->thumbnailUri = $thumbnail;
$item->content = $content;
$item->title = $title;
$item->author = $author;
$item->timestamp = $date;
$item['uri'] = $this->uri.$post->find('abbr')[0]->parent()->getAttribute('href');
$item['content'] = $content;
$item['title'] = $title;
$item['author'] = $author;
$item['timestamp'] = $date;
$this->items[] = $item;
}
}
}
}
public function setDatas(array $param){
if (isset($param['captcha_response']))
unset($param['captcha_response']);
parent::setDatas($param);
}
public function getName() {
return (isset($this->name) ? $this->name.' - ' : '').'Facebook Bridge';
}
public function getURI() {
return 'http://facebook.com';
return (isset($this->authorName) ? $this->authorName.' - ' : '').'Facebook Bridge';
}
public function getCacheDuration() {

37
bridges/FierPandaBridge.php
View File

@ -1,40 +1,25 @@
<?php
Class FierPandaBridge extends BridgeAbstract{
class FierPandaBridge extends BridgeAbstract {
public function loadMetadatas() {
public $maintainer = "snroki";
public $name = "Fier Panda Bridge";
public $uri = "http://www.fier-panda.fr/";
public $description = "Returns latest articles from Fier Panda.";
$this->maintainer = "snroki";
$this->name = "Fier Panda Bridge";
$this->uri = "http://www.fier-panda.fr/";
$this->description = "Returns latest articles from Fier Panda.";
$this->update = "2015-12-11";
}
public function collectData(array $param){
$link = 'http://www.fier-panda.fr/';
$html = $this->file_get_html($link) or $this->returnError('Could not request Fier Panda.', 404);
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri) or $this->returnServerError('Could not request Fier Panda.');
foreach($html->find('div.container-content article') as $element) {
$item = new \Item();
$item->uri = $this->getURI().$element->find('a', 0)->href;
$item->title = trim($element->find('h2 a', 0)->innertext);
$item = array();
$item['uri'] = $this->getURI().$element->find('a', 0)->href;
$item['title'] = trim($element->find('h2 a', 0)->innertext);
// Remove the link at the end of the article
$element->find('p a', 0)->outertext = '';
$item->content = $element->find('p', 0)->innertext;
$item['content'] = $element->find('p', 0)->innertext;
$this->items[] = $item;
}
}
public function getName(){
return 'Fier Panda';
}
public function getURI(){
return 'http://www.fier-panda.fr';
}
public function getCacheDuration(){
return 21600; // 6 hours
}

55
bridges/FlickrExploreBridge.php
View File

@ -1,37 +1,44 @@
<?php
class FlickrExploreBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "sebsauvage";
public $name = "Flickr Explore";
public $uri = "https://www.flickr.com/";
public $description = "Returns the latest interesting images from Flickr";
$this->maintainer = "sebsauvage";
$this->name = "Flickr Explore";
$this->uri = "http://www.flickr.com/explore";
$this->description = "Returns the latest interesting images from Flickr";
$this->update = "2014-05-25";
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri.'explore')
or $this->returnServerError('Could not request Flickr.');
}
foreach($html->find('.photo-list-photo-view') as $element) {
// Get the styles
$style = explode(';', $element->style);
// Get the background-image style
$backgroundImage = explode(':', end($style));
// URI type : url(//cX.staticflickr.com/X/XXXXX/XXXXXXXXX.jpg)
$imageURI = trim(str_replace(['url(', ')'], '', end($backgroundImage)));
// Get the image ID
$imageURIs = explode('_', basename($imageURI));
$imageID = reset($imageURIs);
public function collectData(array $param){
$html = $this->file_get_html('http://www.flickr.com/explore') or $this->returnError('Could not request Flickr.', 404);
foreach($html->find('span.photo_container') as $element) {
$item = new \Item();
$item->uri = 'http://flickr.com'.$element->find('a',0)->href;
$item->thumbnailUri = $element->find('img',0)->getAttribute('data-defer-src');
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" /></a>'; // FIXME: Filter javascript ?
$item->title = $element->find('a',0)->title;
// Get the image JSON via Flickr API
$imageJSON = json_decode($this->getContents(
'https://api.flickr.com/services/rest/?'
.'method=flickr.photos.getInfo&'
.'api_key=103b574d49bd51f0e18bfe907da44a0f&'
.'photo_id='.$imageID.'&'
.'format=json&'
.'nojsoncallback=1'
)) or $this->returnServerError('Could not request Flickr.'); // FIXME: Request time too long...
$item = array();
$item['uri'] = $this->uri.'photo.gne?id='.$imageID;
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $imageURI . '" /></a>'; // FIXME: Filter javascript ?
$item['title'] = $imageJSON->photo->title->_content;
$this->items[] = $item;
}
}
public function getName(){
return 'Flickr Explore';
}
public function getURI(){
return 'http://www.flickr.com/explore';
}
public function getCacheDuration(){
return 21600; // 6 hours
}

81
bridges/FlickrTagBridge.php
View File

@ -1,64 +1,49 @@
<?php
class FlickrTagBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "erwang";
public $name = "Flickr TagUser";
public $uri = "http://www.flickr.com/";
public $description = "Returns the tagged or user images from Flickr";
$this->maintainer = "erwang";
$this->name = "Flickr TagUser";
$this->uri = "http://www.flickr.com/";
$this->description = "Returns the tagged or user images from Flickr";
$this->update = "2014-05-26";
public $parameters = array(
'By keyword' => array(
'q'=>array(
'name'=>'keyword',
'required'=>true
)
),
$this->parameters["By keyword"] =
'[
{
"name" : "Keyword",
"identifier" : "q"
}
]';
'By username' => array(
'u'=>array(
'name'=>'Username',
'required'=>true
)
),
);
$this->parameters["By username"] =
'[
{
"name" : "Username",
"identifier" : "u"
}
]';
}
public function collectData(array $param){
$html = $this->file_get_html('http://www.flickr.com/search/?q=vendee&s=rec') or $this->returnError('Could not request Flickr.', 404);
if (isset($param['q'])) { /* keyword search mode */
$this->request = $param['q'];
$html = $this->file_get_html('http://www.flickr.com/search/?q='.urlencode($this->request).'&s=rec') or $this->returnError('No results for this query.', 404);
}
elseif (isset($param['u'])) { /* user timeline mode */
$this->request = $param['u'];
$html = $this->file_get_html('http://www.flickr.com/photos/'.urlencode($this->request).'/') or $this->returnError('Requested username can\'t be found.', 404);
}
else {
$this->returnError('You must specify a keyword or a Flickr username.', 400);
public function collectData(){
switch($this->queriedContext){
case 'By keyword':
$html = $this->getSimpleHTMLDOM($this->uri.'search/?q='.urlencode($this->getInput('q')).'&s=rec')
or $this->returnServerError('No results for this query.');
break;
case 'by username':
$html = $this->getSimpleHTMLDOM($this->uri.'photos/'.urlencode($this->getInput('u')).'/')
or $this->returnServerError('Requested username can\'t be found.');
break;
}
foreach($html->find('span.photo_container') as $element) {
$item = new \Item();
$item->uri = 'http://flickr.com'.$element->find('a',0)->href;
$item->thumbnailUri = $element->find('img',0)->getAttribute('data-defer-src');
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" /></a>'; // FIXME: Filter javascript ?
$item->title = $element->find('a',0)->title;
$item = array();
$item['uri'] = $this->uri.$element->find('a',0)->href;
$thumbnailUri = $element->find('img',0)->getAttribute('data-defer-src');
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $thumbnailUri . '" /></a>'; // FIXME: Filter javascript ?
$item['title'] = $element->find('a',0)->title;
$this->items[] = $item;
}
}
public function getName(){
return 'Flickr Tag';
}
public function getURI(){
return 'http://www.flickr.com/search/';
}
public function getCacheDuration(){
return 21600; // 6 hours
}

56
bridges/FootitoBridge.php
View File

@ -1,22 +1,18 @@
<?php
class FootitoBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "superbaillot.net";
public $name = "Footito";
public $uri = "http://www.footito.fr/";
public $description = "Footito";
$this->maintainer = "superbaillot.net";
$this->name = "Footito";
$this->uri = "http://www.footito.fr/";
$this->description = "Footito";
$this->update = "21/11/2013";
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Could not request Footito.');
}
public function collectData(array $param){
$html = $this->file_get_html('http://www.footito.fr/') or $this->returnError('Could not request Footito.', 404);
foreach($html->find('div.post') as $element) {
$item = new Item();
$item = array();
$content = trim($element->innertext);
$content = str_replace("<img", "<img style='float : left;'", $content );
$content = str_replace("class=\"logo\"", "style='float : left;'", $content );
@ -27,34 +23,18 @@ class FootitoBridge extends BridgeAbstract{
$content = str_replace("class=\"report-abuse-button\"", "style='display : none;'", $content );
$content = str_replace("class=\"reaction clearfix\"", "style='margin : 10px 0px; padding : 5px; border-bottom : 1px #DDD solid;'", $content );
$content = str_replace("class=\"infos\"", "style='font-size : 0.7em;'", $content );
$item->content = $content;
$item['content'] = $content;
$title = $element->find('.contenu .texte ', 0)->plaintext;
$item->title = $title;
$item['title'] = $title;
$info = $element->find('div.infos', 0);
$item->timestamp = strtotime($info->find('time', 0)->datetime);
$item->name = $info->find('a.auteur', 0)->plaintext;
$item['timestamp'] = strtotime($info->find('time', 0)->datetime);
$item['author'] = $info->find('a.auteur', 0)->plaintext;
$this->items[] = $item;
}
}
public function getName(){
return 'footito';
}
public function getURI(){
return 'http://www.footito.fr/';
}
public function getCacheDuration(){
return 3600; // 1h hours
}
public function getDescription(){
return "Footito via rss-bridge";
}
}
?>

76
bridges/FourchanBridge.php
View File

@ -1,72 +1,58 @@
<?php
class FourchanBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "mitsukarenai";
public $name = "4chan";
public $uri = "https://www.4chan.org/";
public $description = "Returns posts from the specified thread";
$this->maintainer = "mitsukarenai";
$this->name = "4chan";
$this->uri = "https://www.4chan.org/";
$this->description = "Returns posts from the specified thread";
$this->update = "2015-02-01";
public $parameters = array( array(
't'=>array(
'name'=>'Thread URL',
'pattern'=>'(https:\/\/)?boards\.4chan\.org\/.*thread\/.*',
'required'=>true
)
));
$this->parameters[] =
'[
{
"name" : "Thread URL",
"identifier" : "t"
}
]';
}
public function collectData(){
public function collectData(array $param){
if (!isset($param['t']))
$this->returnError('You must specify the thread URL (?t=...)', 400);
$thread = parse_url($param['t']) or $this->returnError('This URL seems malformed, please check it.', 400);
$thread = parse_url($this->getInput('t'))
or $this->returnClientError('This URL seems malformed, please check it.');
if($thread['host'] !== 'boards.4chan.org')
$this->returnError('4chan thread URL only.', 400);
$this->returnClientError('4chan thread URL only.');
if(strpos($thread['path'], 'thread/') === FALSE)
$this->returnError('You must specify the thread URL.', 400);
$this->returnClientError('You must specify the thread URL.');
$url = 'https://boards.4chan.org'.$thread['path'].'';
$html = $this->file_get_html($url) or $this->returnError("Could not request 4chan, thread not found", 404);
$url = 'https://boards.4chan.org'.$thread['path'];
$html = $this->getSimpleHTMLDOM($url)
or $this->returnServerError("Could not request 4chan, thread not found");
foreach($html->find('div.postContainer') as $element) {
$item = new \Item();
$item->id = $element->find('.post', 0)->getAttribute('id');
$item->uri = $url.'#'.$item->id;
$item->timestamp = $element->find('span.dateTime', 0)->getAttribute('data-utc');
$item->author = $element->find('span.name', 0)->plaintext;
$item = array();
$item['id'] = $element->find('.post', 0)->getAttribute('id');
$item['uri'] = $url.'#'.$item['id'];
$item['timestamp'] = $element->find('span.dateTime', 0)->getAttribute('data-utc');
$item['author'] = $element->find('span.name', 0)->plaintext;
if(!empty($element->find('.file', 0) ) ) {
$item->image = $element->find('.file a', 0)->href;
$item->imageThumb = $element->find('.file img', 0)->src;
if(empty($item->imageThumb) and strpos($item->image, '.swf') !== FALSE)
$item->imageThumb = 'http://i.imgur.com/eO0cxf9.jpg';
$item['image'] = $element->find('.file a', 0)->href;
$item['imageThumb'] = $element->find('.file img', 0)->src;
if(!isset($item['imageThumb']) and strpos($item['image'], '.swf') !== FALSE)
$item['imageThumb'] = 'http://i.imgur.com/eO0cxf9.jpg';
}
if(!empty($element->find('span.subject', 0)->innertext )) {
$item->subject = $element->find('span.subject', 0)->innertext;
$item['subject'] = $element->find('span.subject', 0)->innertext;
}
$item->title = (!empty($item->subject) ? $item->subject.' - ' : '' ) . 'reply '.$item->id.' | '.$item->author;
$item['title'] = (isset($item['subject']) ? $item['subject'].' - ' : '' ) . 'reply '.$item['id'].' | '.$item['author'];
$item->content = (!empty($item->image) ? '<a href="'.$item->image.'"><img alt="'.$item->id.'" src="'.$item->imageThumb.'" /></a><br>' : '') . '<span id="'.$item->id.'">'.$element->find('.postMessage', 0)->innertext.'</span>';
$item['content'] = (isset($item['image']) ? '<a href="'.$item['image'].'"><img alt="'.$item['id'].'" src="'.$item['imageThumb'].'" /></a><br>' : '') . '<span id="'.$item['id'].'">'.$element->find('.postMessage', 0)->innertext.'</span>';
$this->items[] = $item;
}
$this->items = array_reverse($this->items);
}
public function getName(){
return '4chan';
}
public function getURI(){
return 'https://www.4chan.org/';
}
public function getCacheDuration(){
return 300; // 5min

69
bridges/FrandroidBridge.php
View File

@ -1,69 +0,0 @@
<?php
class FrandroidBridge extends BridgeAbstract
{
public function loadMetadatas() {
$this->maintainer = "Daiyousei";
$this->name = "Frandroid";
$this->uri = "http://www.frandroid.com/";
$this->description = "Returns the RSS feed from Frandroid (full text articles)";
$this->update = "2015-03-05";
}
public function collectData(array $param)
{
function FrandroidStripCDATA($string)
{
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function FrandroidExtractContent($url)
{
$html2 = $this->file_get_html($url);
$html3 = $html2->find('div.post-content', 0);
$html3->find('div.no-sidebar-ad-top', 0)->outertext = '';
$ret = $html3->find('div.shortcode-container');
foreach ($ret as $value) {
$value->outertext = '';
}
$html3->find('div#hrr-link', 0)->outertext = '';
$text = $html3->innertext;
$text = strip_tags($text, '<h1><span><h2><p><b><a><blockquote><img><em><ul><ol>');
return $text;
}
$html = $this->file_get_html('http://feeds.feedburner.com/Frandroid?format=xml') or $this->returnError('Could not request Frandroid.', 404);
$limit = 0;
foreach ($html->find('item') as $element) {
if ($limit < 5) {
$item = new \Item();
$item->title = FrandroidStripCDATA($element->find('title', 0)->innertext);
$item->uri = FrandroidStripCDATA($element->find('guid', 0)->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = FrandroidExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
}
}
public function getName()
{
return 'Frandroid';
}
public function getURI()
{
return 'http://www.frandroid.com/';
}
public function getCacheDuration()
{
return 300; // 5min
}
}

46
bridges/Freenews.php
View File

@ -1,46 +0,0 @@
<?php
define("FREENEWS_RSS", 'http://feeds.feedburner.com/Freenews-Freebox?format=xml');
class Freenews extends RssExpander {
public function loadMetadatas() {
$this->maintainer = "mitsukarenai";
$this->name = "Freenews";
$this->uri = "http://freenews.fr";
$this->description = "Un site d'actualité pour les freenautes (mais ne parlant pas que de la freebox). Ne rentrez pas d'id si vous voulez accéder aux actualités générales.";
$this->update = "26/03/2014";
$this->parameters[] =
'[
{
"name" : "Id de la rubrique (sans le \'-\')",
"identifier" : "id"
}
]';
}
public function collectData(array $param){
parent::collectExpandableDatas($param, FREENEWS_RSS);
}
protected function parseRSSItem($newsItem) {
$item = new Item();
$item->title = trim($newsItem->title);
// $this->message("item has for title \"".$item->title."\"");
if(empty($newsItem->guid)) {
$item->uri = (string) $newsItem->link;
} else {
$item->uri = (string) $newsItem->guid;
}
// now load that uri from cache
// $this->message("now loading page ".$item->uri);
$articlePage = str_get_html($this->get_cached($item->uri));
$content = $articlePage->find('.post-container', 0);
$item->content = $content->innertext;
$item->name = $articlePage->find('a[rel=author]', 0)->innertext;
// format should parse 2014-03-25T16:21:20Z. But, according to http://stackoverflow.com/a/10478469, it is not that simple
$item->timestamp = $this->RSS_2_0_time_to_timestamp($newsItem);
return $item;
}
}

34
bridges/FreenewsBridge.php Normal file
View File

@ -0,0 +1,34 @@
<?php
define("FREENEWS_RSS", 'http://feeds.feedburner.com/Freenews-Freebox?format=xml');
class FreenewsBridge extends RssExpander {
public $maintainer = "mitsukarenai";
public $name = "Freenews";
public $uri = "http://freenews.fr";
public $description = "Un site d'actualité pour les freenautes (mais ne parlant pas que de la freebox). Ne rentrez pas d'id si vous voulez accéder aux actualités générales.";
public function collectData(){
parent::collectExpandableDatas(FREENEWS_RSS);
}
protected function parseRSSItem($newsItem) {
$item = array();
$item['title'] = trim($newsItem->title);
$this->debugMessage("item has for title \"".$item['title']."\"");
if(empty($newsItem->guid)) {
$item['uri'] = (string) $newsItem->link;
} else {
$item['uri'] = (string) $newsItem->guid;
}
// now load that uri from cache
$this->debugMessage("now loading page ".$item['uri']);
$articlePage = $this->get_cached($item['uri']);
$content = $articlePage->find('.post-container', 0);
$item['content'] = $content->innertext;
$item['author'] = $articlePage->find('a[rel=author]', 0)->innertext;
// format should parse 2014-03-25T16:21:20Z. But, according to http://stackoverflow.com/a/10478469, it is not that simple
$item['timestamp'] = $this->RSS_2_0_time_to_timestamp($newsItem);
return $item;
}
}

239
bridges/FuturaSciencesBridge.php
View File

@ -1,115 +1,84 @@
<?php
class FuturaSciencesBridge extends BridgeAbstract {
public function loadMetadatas() {
public $maintainer = 'ORelio';
public $name = 'Futura-Sciences Bridge';
public $uri = 'http://www.futura-sciences.com/';
public $description = 'Returns the newest articles.';
$this->maintainer = 'ORelio';
$this->name = $this->getName();
$this->uri = $this->getURI();
$this->description = 'Returns the newest articles.';
$this->update = '2016-03-20';
public $parameters = array( array(
'feed'=> array(
'name'=>'Feed',
'type'=>'list',
'values'=>array(
'Les flux multi-magazines'=>array(
'Les dernières actualités de Futura-Sciences'=>'actualites',
'Les dernières définitions de Futura-Sciences'=>'definitions',
'Les dernières photos de Futura-Sciences'=>'photos',
'Les dernières questions - réponses de Futura-Sciences'=>'questions-reponses',
'Les derniers dossiers de Futura-Sciences'=>'dossiers'
),
'Les flux Services'=> array(
'Les cartes virtuelles de Futura-Sciences'=>'services/cartes-virtuelles',
'Les fonds d\'écran de Futura-Sciences'=>'services/fonds-ecran'
),
'Les flux Santé'=>array(
'Les dernières actualités de Futura-Santé'=>'sante/actualites',
'Les dernières définitions de Futura-Santé'=>'sante/definitions',
'Les dernières questions-réponses de Futura-Santé'=>'sante/question-reponses',
'Les derniers dossiers de Futura-Santé'=>'sante/dossiers'
),
'Les flux High-Tech'=>array(
'Les dernières actualités de Futura-High-Tech'=>'high-tech/actualites',
'Les dernières astuces de Futura-High-Tech'=>'high-tech/question-reponses',
'Les dernières définitions de Futura-High-Tech'=>'high-tech/definitions',
'Les derniers dossiers de Futura-High-Tech'=>'high-tech/dossiers'
),
'Les flux Espace'=>array(
'Les dernières actualités de Futura-Espace'=>'espace/actualites',
'Les dernières définitions de Futura-Espace'=>'espace/definitions',
'Les dernières questions-réponses de Futura-Espace'=>'espace/question-reponses',
'Les derniers dossiers de Futura-Espace'=>'espace/dossiers'
),
'Les flux Environnement'=>array(
'Les dernières actualités de Futura-Environnement'=>'environnement/actualites',
'Les dernières définitions de Futura-Environnement'=>'environnement/definitions',
'Les dernières questions-réponses de Futura-Environnement'=>'environnement/question-reponses',
'Les derniers dossiers de Futura-Environnement'=>'environnement/dossiers'
),
'Les flux Maison'=>array(
'Les dernières actualités de Futura-Maison'=>'maison/actualites',
'Les dernières astuces de Futura-Maison'=>'maison/question-reponses',
'Les dernières définitions de Futura-Maison'=>'maison/definitions',
'Les derniers dossiers de Futura-Maison'=>'maison/dossiers'
),
'Les flux Nature'=>array(
'Les dernières actualités de Futura-Nature'=>'nature/actualites',
'Les dernières définitions de Futura-Nature'=>'nature/definitions',
'Les dernières questions-réponses de Futura-Nature'=>'nature/question-reponses',
'Les derniers dossiers de Futura-Nature'=>'nature/dossiers'
),
'Les flux Terre'=>array(
'Les dernières actualités de Futura-Terre'=>'terre/actualites',
'Les dernières définitions de Futura-Terre'=>'terre/definitions',
'Les dernières questions-réponses de Futura-Terre'=>'terre/question-reponses',
'Les derniers dossiers de Futura-Terre'=>'terre/dossiers'
),
'Les flux Matière'=>array(
'Les dernières actualités de Futura-Matière'=>'matiere/actualites',
'Les dernières définitions de Futura-Matière'=>'matiere/definitions',
'Les dernières questions-réponses de Futura-Matière'=>'matiere/question-reponses',
'Les derniers dossiers de Futura-Matière'=>'matiere/dossiers'
),
'Les flux Mathématiques'=>array(
'Les dernières actualités de Futura-Mathématiques'=>'mathematiques/actualites',
'Les derniers dossiers de Futura-Mathématiques'=>'mathematiques/dossiers'
)
)
)
));
$this->parameters[] =
'[
{
"name" : "Feed",
"type" : "list",
"identifier" : "feed",
"values" :
[
{ "name" : "---- Select ----", "value" : "" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux multi-magazines", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura-Sciences", "value" : "actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières définitions de Futura-Sciences", "value" : "definitions" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières photos de Futura-Sciences", "value" : "photos" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières questions - réponses de Futura-Sciences", "value" : "questions-reponses" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura-Sciences", "value" : "dossiers" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux Services", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les cartes virtuelles de Futura-Sciences", "value" : "services/cartes-virtuelles" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les fonds d\'écran de Futura-Sciences", "value" : "services/fonds-ecran" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux Santé", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura-Santé", "value" : "sante/actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières définitions de Futura-Santé", "value" : "sante/definitions" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières questions-réponses de Futura-Santé", "value" : "sante/questions-reponses" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura-Santé", "value" : "sante/dossiers" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux High-Tech", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura High-Tech", "value" : "high-tech/actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières astuces de Futura High-Tech", "value" : "high-tech/questions-reponses" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières définitions de Futura High-Tech", "value" : "high-tech/definitions" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura High-Tech", "value" : "high-tech/dossiers" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux Espace", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura-Espace", "value" : "espace/actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières définitions de Futura-Espace", "value" : "espace/definitions" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières questions-réponses de Futura-Espace", "value" : "espace/questions-reponses" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura-Espace", "value" : "espace/dossiers" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux Environnement", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura-Environnement", "value" : "environnement/actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières définitions de Futura-Environnement", "value" : "environnement/definitions" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières questions - réponses de Futura-Environnement", "value" : "environnement/questions-reponses" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura-Environnement", "value" : "environnement/dossiers" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux Maison", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura-Maison", "value" : "maison/actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières astuces de Futura-Maison", "value" : "maison/questions-reponses" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières définitions de Futura-Maison", "value" : "maison/definitions" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura-Maison", "value" : "maison/dossiers" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux Nature", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura-Nature", "value" : "nature/actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières définitions de Futura-Nature", "value" : "nature/definitions" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières questions-réponses de Futura-Nature", "value" : "nature/questions-reponses" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura-Nature", "value" : "nature/dossiers" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux Terre", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura-Terre", "value" : "terre/actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières définitions de Futura-Terre", "value" : "terre/definitions" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières questions-réponses de Futura-Terre", "value" : "terre/questions-reponses" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura-Terre", "value" : "terre/dossiers" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux Matière", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura-Matière", "value" : "matiere/actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières définitions de Futura-Matière", "value" : "matiere/definitions" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières questions-réponses de Futura-Matière", "value" : "matiere/questions-reponses" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura-Matière", "value" : "matiere/dossiers" },
{ "name" : "", "value" : "" },
{ "name" : "Les flux Mathématiques", "value" : "" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les dernières actualités de Futura-Mathématiques", "value" : "mathematiques/actualites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Les derniers dossiers de Futura-Mathématiques", "value" : "mathematiques/dossiers" }
]
}
]';
}
public function collectData(array $param) {
public function collectData(){
function StripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
@ -149,23 +118,34 @@ class FuturaSciencesBridge extends BridgeAbstract {
return $string;
}
if (empty($param['feed']))
$this->returnError('Please select a feed to display.'.$url, 400);
if ($param['feed'] !== preg_replace('/[^a-zA-Z-\/]+/', '', $param['feed']) || substr_count($param['feed'], '/') > 1 || strlen($param['feed'] > 64))
$this->returnError('Invalid "feed" parameter.'.$url, 400);
// Extracts the author from an article or element
function ExtractAuthor($article, $element){
$article_author = $article->find('span.author', 0);
if($article_author){
$authorname = trim(str_replace(', Futura-Sciences', '', $article_author->plaintext));
if(empty($authorname)){
$element_author = $element->find('author', 0);
if($element_author)
$authorname = StripCDATA($element_author->plaintext);
else
return '';
}
return $authorname;
}
return '';
}
$url = $this->getURI().'rss/'.$param['feed'].'.xml';
$html = $this->file_get_html($url) or $this->returnError('Could not request Futura-Sciences: '.$url, 500);
$url = $this->getURI().'rss/'.$this->getInput('feed').'.xml';
$html = $this->getSimpleHTMLDOM($url)
or $this->returnServerError('Could not request Futura-Sciences: '.$url);
$limit = 0;
foreach($html->find('item') as $element) {
if ($limit < 10) {
$article_url = str_replace('#xtor=RSS-8', '', StripCDATA($element->find('guid', 0)->plaintext));
$article = $this->file_get_html($article_url) or $this->returnError('Could not request Futura-Sciences: '.$article_url, 500);
$article = $this->getSimpleHTMLDOM($article_url) or $this->returnServerError('Could not request Futura-Sciences: '.$article_url);
$contents = $article->find('div.content', 0)->innertext;
$author = trim(str_replace(', Futura-Sciences', '', $article->find('span.author', 0)->plaintext));
if (empty($author))
$author = StripCDATA($element->find('author', 0)->plaintext);
foreach (array(
'<div class="clear',
@ -191,29 +171,16 @@ class FuturaSciencesBridge extends BridgeAbstract {
$contents = StripWithDelimiters($contents, 'fs:xt:clicktype="', '"');
$contents = StripWithDelimiters($contents, 'fs:xt:clickname="', '"');
$item = new \Item();
$item->author = $author;
$item->uri = $article_url;
$item->title = StripCDATA($element->find('title', 0)->innertext);
$item->thumbnailUri = StripCDATA($element->find('enclosure', 0)->url);
$item->timestamp = strtotime(StripCDATA($element->find('pubDate', 0)->plaintext));
$item->content = trim($contents);
$item = array();
$item['author'] = ExtractAuthor($article, $element);
$item['uri'] = $article_url;
$item['title'] = StripCDATA($element->find('title', 0)->innertext);
$item['timestamp'] = strtotime(StripCDATA($element->find('pubDate', 0)->plaintext));
$item['content'] = trim($contents);
$this->items[] = $item;
$limit++;
}
}
}
public function getName() {
return 'Futura-Sciences Bridge';
}
public function getURI() {
return 'http://www.futura-sciences.com/';
}
public function getCacheDuration() {
return 3600;
}
}

196
bridges/GBAtempBridge.php
View File

@ -1,153 +1,125 @@
<?php
class GBAtempBridge extends BridgeAbstract {
private $filter = '';
public $maintainer = 'ORelio';
public $name = 'GBAtemp';
public $uri = 'http://gbatemp.net/';
public $description = 'GBAtemp is a user friendly underground video game community.';
public function loadMetadatas() {
public $parameters = array( array(
'type'=>array(
'name'=>'Type',
'type'=>'list',
'required'=>true,
'values'=>array(
'News'=>'N',
'Reviews'=>'R',
'Tutorials'=>'T',
'Forum'=>'F'
)
)
));
$this->maintainer = 'ORelio';
$this->name = 'GBAtemp';
$this->uri = $this->getURI();
$this->description = 'GBAtemp is a user friendly underground video game community.';
$this->update = '2016-02-14';
$this->parameters[] =
'[
{
"name" : "Type",
"type" : "list",
"identifier" : "type",
"values" :
[
{
"name" : "News",
"value" : "N"
},
{
"name" : "Reviews",
"value" : "R"
},
{
"name" : "Tutorials",
"value" : "T"
},
{
"name" : "Forum",
"value" : "F"
}
]
}
]';
private function ExtractFromDelimiters($string, $start, $end) {
if (strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
} return false;
}
public function collectData(array $param) {
private function StripWithDelimiters($string, $start, $end) {
while (strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
} return $string;
}
function ExtractFromDelimiters($string, $start, $end) {
if (strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
} return false;
private function build_item($uri, $title, $author, $timestamp, $content) {
$item = array();
$item['uri'] = $uri;
$item['title'] = $title;
$item['author'] = $author;
$item['timestamp'] = $timestamp;
$item['content'] = $content;
return $item;
}
private function cleanup_post_content($content, $site_url) {
$content = str_replace(':arrow:', '&#x27a4;', $content);
$content = str_replace('href="attachments/', 'href="'.$site_url.'attachments/', $content);
$content = $this->StripWithDelimiters($content, '<script', '</script>');
return $content;
}
private function fetch_post_content($uri, $site_url) {
$html = $this->getSimpleHTMLDOM($uri);
if(!$html){
return 'Could not request GBAtemp '.$uri;
}
function StripWithDelimiters($string, $start, $end) {
while (strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
} return $string;
}
$content = $html->find('div.messageContent', 0)->innertext;
return $this->cleanup_post_content($content, $site_url);
}
function build_item($uri, $title, $author, $timestamp, $thumnail, $content) {
$item = new \Item();
$item->uri = $uri;
$item->title = $title;
$item->author = $author;
$item->timestamp = $timestamp;
$item->thumbnailUri = $thumnail;
$item->content = $content;
return $item;
}
public function collectData(){
function cleanup_post_content($content, $site_url) {
$content = str_replace(':arrow:', '&#x27a4;', $content);
$content = str_replace('href="attachments/', 'href="'.$site_url.'attachments/', $content);
$content = StripWithDelimiters($content, '<script', '</script>');
return $content;
}
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Could not request GBAtemp.');
function fetch_post_content($uri, $site_url) {
$html = $this->file_get_html($uri) or $this->returnError('Could not request GBAtemp: '.$uri, 500);
$content = $html->find('div.messageContent', 0)->innertext;
return cleanup_post_content($content, $site_url);
}
$typeFilter = '';
if (!empty($param['type'])) {
if ($param['type'] == 'N' || $param['type'] == 'R' || $param['type'] == 'T' || $param['type'] == 'F') {
$typeFilter = $param['type'];
if ($typeFilter == 'N') { $this->filter = 'News'; }
if ($typeFilter == 'R') { $this->filter = 'Review'; }
if ($typeFilter == 'T') { $this->filter = 'Tutorial'; }
if ($typeFilter == 'F') { $this->filter = 'Forum'; }
} else $this->returnError('The provided type filter is invalid. Expecting N, R, T, or F.', 400);
} else $this->returnError('Please provide a type filter. Expecting N, R, T, or F.', 400);
$html = $this->file_get_html($this->getURI()) or $this->returnError('Could not request GBAtemp.', 500);
if ($typeFilter == 'N') {
switch($this->getInput('type')){
case 'N':
foreach ($html->find('li[class=news_item full]') as $newsItem) {
$url = $this->getURI().$newsItem->find('a', 0)->href;
$img = $this->getURI().$newsItem->find('img', 0)->src;
$time = intval(ExtractFromDelimiters($newsItem->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$url = $this->uri.$newsItem->find('a', 0)->href;
$time = intval($this->ExtractFromDelimiters($newsItem->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$author = $newsItem->find('a.username', 0)->plaintext;
$title = $newsItem->find('a', 1)->plaintext;
$content = fetch_post_content($url, $this->getURI());
$this->items[] = build_item($url, $title, $author, $time, $img, $content);
$content = $this->fetch_post_content($url, $this->uri);
$this->items[] = $this->build_item($url, $title, $author, $time, $content);
}
} else if ($typeFilter == 'R') {
case 'R':
foreach ($html->find('li.portal_review') as $reviewItem) {
$url = $this->getURI().$reviewItem->find('a', 0)->href;
$img = $this->getURI().ExtractFromDelimiters($reviewItem->find('a', 0)->style, 'image:url(', ')');
$url = $this->uri.$reviewItem->find('a', 0)->href;
$title = $reviewItem->find('span.review_title', 0)->plaintext;
$content = $this->file_get_html($url) or $this->returnError('Could not request GBAtemp: '.$uri, 500);
$content = $this->getSimpleHTMLDOM($url) or $this->returnServerError('Could not request GBAtemp: '.$uri);
$author = $content->find('a.username', 0)->plaintext;
$time = intval(ExtractFromDelimiters($content->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$time = intval($this->ExtractFromDelimiters($content->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$intro = '<p><b>'.($content->find('div#review_intro', 0)->plaintext).'</b></p>';
$review = $content->find('div#review_main', 0)->innertext;
$subheader = '<p><b>'.$content->find('div.review_subheader', 0)->plaintext.'</b></p>';
$procons = $content->find('table.review_procons', 0)->outertext;
$scores = $content->find('table.reviewscores', 0)->outertext;
$content = cleanup_post_content($intro.$review.$subheader.$procons.$scores, $this->getURI());
$this->items[] = build_item($url, $title, $author, $time, $img, $content);
$content = $this->cleanup_post_content($intro.$review.$subheader.$procons.$scores, $this->uri);
$this->items[] = $this->build_item($url, $title, $author, $time, $content);
}
} else if ($typeFilter == 'T') {
case 'T':
foreach ($html->find('li.portal-tutorial') as $tutorialItem) {
$url = $this->getURI().$tutorialItem->find('a', 0)->href;
$url = $this->uri.$tutorialItem->find('a', 0)->href;
$title = $tutorialItem->find('a', 0)->plaintext;
$time = intval(ExtractFromDelimiters($tutorialItem->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$time = intval($this->ExtractFromDelimiters($tutorialItem->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$author = $tutorialItem->find('a.username', 0)->plaintext;
$content = fetch_post_content($url, $this->getURI());
$this->items[] = build_item($url, $title, $author, $time, '', $content);
$content = $this->fetch_post_content($url, $this->uri);
$this->items[] = $this->build_item($url, $title, $author, $time, $content);
}
} else if ($typeFilter == 'F') {
case 'F':
foreach ($html->find('li.rc_item') as $postItem) {
$url = $this->getURI().$postItem->find('a', 1)->href;
$url = $this->uri.$postItem->find('a', 1)->href;
$title = $postItem->find('a', 1)->plaintext;
$time = intval(ExtractFromDelimiters($postItem->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$time = intval($this->ExtractFromDelimiters($postItem->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$author = $postItem->find('a.username', 0)->plaintext;
$content = fetch_post_content($url, $this->getURI());
$this->items[] = build_item($url, $title, $author, $time, '', $content);
$content = $this->fetch_post_content($url, $this->uri);
$this->items[] = $this->build_item($url, $title, $author, $time, $content);
}
}
}
public function getName() {
return 'GBAtemp'.(empty($this->filter) ? '' : ' '.$this->filter).' Bridge';
}
public function getURI() {
return 'http://gbatemp.net/';
$type=array_search(
$this->getInput('type'),
$this->parameters[$this->queriedContext]['type']['values']
);
return 'GBAtemp '.$type.' Bridge';
}
public function getCacheDuration() {

73
bridges/Gawker.php
View File

@ -1,73 +0,0 @@
<?php
define("RSS_PREFIX", "http://feeds.gawker.com/");
define("RSS_SUFFIX", "/full");
class Gawker extends RssExpander{
public function loadMetadatas() {
$this->maintainer = "mitsukarenai";
$this->name = "Gawker media";
$this->uri = "http://feeds.gawker.com/";
$this->description = "A bridge allowing access to any of the numerous Gawker media blogs (Lifehacker, deadspin, Kotaku, Jezebel, and so on. Notice you have to give its id to find the RSS stream in gawker maze";
$this->update = "27/03/2014";
$this->parameters[] =
'[
{
"name" : "site id to put in uri between feeds.gawker.com and /full .. which is obviously not full AT ALL",
"identifier" : "site"
}
]';
}
private function toURI($name) {
return RSS_PREFIX.$name.RSS_SUFFIX;
}
public function collectData(array $param){
if (empty($param['site'])) {
trigger_error("If no site is provided, nothing is gonna happen", E_USER_ERROR);
} else {
$this->name = $param['site'];
$url = $this->toURI(strtolower($param['site']));
}
// $this->message("loading feed from ".$this->getURI());
parent::collectExpandableDatas($param, $url);
}
protected function parseRSSItem($newsItem) {
$item = new Item();
$item->uri = trim($newsItem->link);
$item->title = trim($newsItem->title);
$item->timestamp = $this->RSS_2_0_time_to_timestamp($newsItem);
// $this->message("///////////////////////////////////////////////////////////////////////////////////////\nprocessing item ".var_export($item, true)."\n\n\nbuilt from\n\n\n".var_export($newsItem, true));
try {
// now load that uri from cache
// $this->message("loading page ".$item->uri);
$articlePage = str_get_html($this->get_cached($item->uri));
if(is_object($articlePage)) {
$content = $articlePage->find('.post-content', 0);
HTMLSanitizer::defaultImageSrcTo($content, $this->getURI());
$vcard = $articlePage->find('.vcard', 0);
if(is_object($vcard)) {
$authorLink = $vcard->find('a', 0);
$item->name = $authorLink->innertext;
// TODO use author link href to fill the feed info
}
// $this->message("item quite loaded : ".var_export($item, true));
// I set item content as last element, for easier var_export reading
$item->content = $content->innertext;
} else {
throw new Exception("cache content for ".$item->uri." is NOT a Simple DOM parser object !");
}
} catch(Exception $e) {
$this->message("obtaining ".$item->uri." resulted in exception ".$e->getMessage().". Deleting cached page ...");
// maybe file is incorrect. it should be discarded from cache
$this->remove_from_cache($item->url);
$item->content = $e->getMessage();
}
return $item;
}
}

76
bridges/GelbooruBridge.php
View File

@ -1,63 +1,39 @@
<?php
class GelbooruBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "mitsukarenai";
public $name = "Gelbooru";
public $uri = "http://gelbooru.com/";
public $description = "Returns images from given page";
$this->maintainer = "mitsukarenai";
$this->name = "Gelbooru";
$this->uri = "http://gelbooru.com/";
$this->description = "Returns images from given page";
$this->update = "2014-05-25";
$this->parameters[] =
'[
{
"name" : "page",
"identifier" : "p",
"type" : "number"
},
{
"name" : "tags",
"identifier" : "t"
}
]';
}
public function collectData(array $param){
$page = 0;
if (isset($param['p'])) {
$page = (int)preg_replace("/[^0-9]/",'', $param['p']);
$page = $page - 1;
$page = $page * 63;
}
if (isset($param['t'])) {
$tags = urlencode($param['t']);
}
$html = $this->file_get_html("http://gelbooru.com/index.php?page=post&s=list&tags=$tags&pid=$page") or $this->returnError('Could not request Gelbooru.', 404);
public $parameters = array( array(
'p'=>array(
'name'=>'page',
'type'=>'number'
),
't'=>array('name'=>'tags')
));
public function collectData(){
$html = $this->getSimpleHTMLDOM(
$this->uri.'index.php?page=post&s=list&'
.'&pid='.($this->getInput('p')?($this->getInput('p') -1)*63:'')
.'&tags='.urlencode($this->getInput('t'))
) or $this->returnServerError('Could not request Gelbooru.');
foreach($html->find('div[class=content] span') as $element) {
$item = new \Item();
$item->uri = 'http://gelbooru.com/'.$element->find('a', 0)->href;
$item->postid = (int)preg_replace("/[^0-9]/",'', $element->getAttribute('id'));
$item->timestamp = time();
$item->thumbnailUri = $element->find('img', 0)->src;
$item->tags = $element->find('img', 0)->getAttribute('alt');
$item->title = 'Gelbooru | '.$item->postid;
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" /></a><br>Tags: '.$item->tags;
$this->items[] = $item;
$item = array();
$item['uri'] = $this->uri.$element->find('a', 0)->href;
$item['postid'] = (int)preg_replace("/[^0-9]/",'', $element->getAttribute('id'));
$item['timestamp'] = time();
$thumbnailUri = $element->find('img', 0)->src;
$item['tags'] = $element->find('img', 0)->getAttribute('alt');
$item['title'] = 'Gelbooru | '.$item['postid'];
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $thumbnailUri . '" /></a><br>Tags: '.$item['tags'];
$this->items[] = $item;
}
}
public function getName(){
return 'Gelbooru';
}
public function getURI(){
return 'http://gelbooru.com/';
}
public function getCacheDuration(){
return 1800; // 30 minutes
}

101
bridges/GiphyBridge.php
View File

@ -3,63 +3,50 @@ define('GIPHY_LIMIT', 10);
class GiphyBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "kraoc";
public $name = "Giphy Bridge";
public $uri = "http://giphy.com/";
public $description = "Bridge for giphy.com";
$this->maintainer = "kraoc";
$this->name = "Giphy Bridge";
$this->uri = "http://giphy.com/";
$this->description = "Bridge for giphy.com";
$this->update = "2014-12-05";
public $parameters = array( array(
's'=>array(
'name'=>'search tag',
'required'=>true
),
'n'=>array(
'name'=>'max number of returned items',
'type'=>'number'
)
));
$this->parameters["By tag"] =
'[
{
"name" : "search tag",
"identifier" : "s"
}
]';
$this->parameters["Without tag"] =
'[
{
"name" : "max number of returned items",
"type" : "number",
"identifier" : "n"
}
]';
}
public function collectData(array $param){
$html = '';
public function collectData(){
$html = '';
$base_url = 'http://giphy.com';
if (isset($param['s'])) { /* keyword search mode */
$html = $this->file_get_html($base_url.'/search/'.urlencode($param['s'].'/')) or $this->returnError('No results for this query.', 404);
}
else {
$this->returnError('You must specify a search worf (?s=...).', 400);
}
$html = $this->getSimpleHTMLDOM($this->uri.'/search/'.urlencode($this->getInput('s').'/'))
or $this->returnServerError('No results for this query.');
$max = GIPHY_LIMIT;
if (isset($param['n'])) {
$max = (integer) $param['n'];
if ($this->getInput('n')) {
$max = $this->getInput('n');
}
$limit = 0;
$kw = urlencode($param['s']);
$kw = urlencode($this->getInput('s'));
foreach($html->find('div.hoverable-gif') as $entry) {
if($limit < $max) {
$node = $entry->first_child();
$href = $node->getAttribute('href');
$html2 = $this->file_get_html($base_url . $href) or $this->returnError('No results for this query.', 404);
$node = $entry->first_child();
$href = $node->getAttribute('href');
$html2 = $this->getSimpleHTMLDOM($this->uri . $href)
or $this->returnServerError('No results for this query.');
$figure = $html2->getElementByTagName('figure');
$img = $figure->firstChild();
$caption = $figure->lastChild();
$item = new \Item();
$item->id = $img->getAttribute('data-gif_id');
$item->uri = $img->getAttribute('data-bitly_gif_url');
$item->username = 'Giphy - '.ucfirst($kw);
$item = array();
$item['id'] = $img->getAttribute('data-gif_id');
$item['uri'] = $img->getAttribute('data-bitly_gif_url');
$item['username'] = 'Giphy - '.ucfirst($kw);
$title = $caption->innertext();
$title = preg_replace('/\s+/', ' ',$title);
$title = str_replace('animated GIF', '', $title);
@ -67,33 +54,21 @@ class GiphyBridge extends BridgeAbstract{
$title = preg_replace('/\s+/', ' ',$title);
$title = trim($title);
if (strlen($title) <= 0) {
$title = $item->id;
$title = $item['id'];
}
$item->title = trim($title);
$item->content =
'<a href="'.$item->uri.'">'
$item['title'] = trim($title);
$item['content'] =
'<a href="'.$item['uri'].'">'
.'<img src="'.$img->getAttribute('src').'" width="'.$img->getAttribute('data-original-width').'" height="'.$img->getAttribute('data-original-height').'" />'
.'</a>';
$this->items[] = $item;
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'Giphy Bridge';
}
public function getURI(){
return 'http://giphy.com/';
}
public function getCacheDuration(){
return 300; // 5 minutes
}
public function getUsername(){
return $this->items[0]->username;
}
}

104
bridges/GithubIssueBridge.php
View File

@ -1,66 +1,72 @@
<?php
/**
* GithubIssueBridge
*
* @name GithubIssue Bridge
* @description Returns the comments of a github project issue
*/
class GithubIssueBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = 'Pierre Mazière';
$this->name = 'Github Issue';
$this->uri = '';
$this->description = 'Returns the comments of a github project issue';
$this->update = '2016-06-25';
public $maintainer = 'Pierre Mazière';
public $name = 'Github Issue';
public $uri = 'https://github.com/';
public $description = 'Returns the issues or comments of an issue of a github project';
$this->parameters[]=
'[
{
"name" : "User name",
"identifier" : "u"
},
{
"name" : "Project name",
"identifier" : "p"
},
{
"name" : "Issue number",
"identifier" : "i"
}
public $parameters=array(
'global'=>array (
'u'=>array(
'name'=>'User name',
'required'=>true
),
'p'=>array(
'name'=>'Project name',
'required'=>true
)
),
]';
}
'Project Issues'=>array(),
'Issue comments'=>array(
'i'=>array(
'name'=>'Issue number',
'type'=>'number',
'required'=>'true'
)
)
);
public function collectData(array $param){
$uri = 'https://github.com/'.$param['u'].'/'.$param['p'].'/issues/'.$param['i'];
$html = file_get_html($uri)
or $this->returnError('No results for Github Issue '.$param['i'].' in project '.$param['u'].'/'.$param['p'], 404);
public function collectData(){
$uri = $this->uri.$this->getInput('u').'/'.$this->getInput('p')
.'/issues/'.$this->getInput('i');
$html = $this->getSimpleHTMLDOM($uri)
or $this->returnServerError('No results for Github Issue '.$this->getInput('i').' in project '.$this->getInput('u').'/'.$this->getInput('p'));
foreach($html->find('.js-comment-container') as $comment){
switch($this->queriedContext){
case 'Issue Comments':
foreach($html->find('.js-comment-container') as $comment){
$item = new \Item();
$item->name=$comment->find('img',0)->getAttribute('alt');
$item = array();
$item['author']=$comment->find('img',0)->getAttribute('alt');
$comment=$comment->firstChild()->nextSibling();
$comment=$comment->firstChild()->nextSibling();
$item->uri=$uri.'#'.$comment->getAttribute('id');
$item->title=trim($comment->firstChild()->plaintext);
$item->timestamp=strtotime($comment->find('relative-time',0)->getAttribute('datetime'));
$item->content=$comment->find('.comment-body',0)->innertext;
$item['uri']=$uri.'#'.$comment->getAttribute('id');
$item['title']=trim($comment->firstChild()->plaintext);
$item['timestamp']=strtotime($comment->find('relative-time',0)->getAttribute('datetime'));
$item['content']=$comment->find('.comment-body',0)->innertext;
$this->items[]=$item;
$this->items[]=$item;
}
break;
case 'Project Issues':
foreach($html->find('.js-active-navigation-container .js-navigation-item') as $issue){
$item=array();
$info=$issue->find('.opened-by',0);
$item['author']=$info->find('a',0)->plaintext;
$item['timestamp']=strtotime($info->find('relative-time',0)->getAttribute('datetime'));
$item['title']=$issue->find('.js-navigation-open',0)->plaintext;
$comments=$issue->find('.col-5',0)->plaintext;
$item['content']='Comments: '.($comments?$comments:'0');
$item['uri']=$this->uri.$issue->find('.js-navigation-open',0)->getAttribute('href');
$this->items[]=$item;
}
break;
}
}
public function getName(){
return 'Github Issue';
}
public function getURI(){
return '';
}
public function getCacheDuration(){
return 600; // ten minutes
}

98
bridges/GitlabCommitsBridge.php
View File

@ -1,94 +1,66 @@
<?php
/**
* GitlabCommitsBridge
*
* @name GitlabCommits Bridge
* @description Returns the commits of a project hosted on a gitlab instance
*/
class GitlabCommitsBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = 'Pierre Mazière';
$this->name = 'Gitlab Commits';
$this->uri = '';
$this->description = 'Returns the commits of a project hosted on a gitlab instance';
$this->update = '2016-06-19';
public $maintainer = 'Pierre Mazière';
public $name = 'Gitlab Commits';
public $uri = '';
public $description = 'Returns the commits of a project hosted on a gitlab instance';
$this->parameters=
'[
{
"name" : "Base URI",
"identifier" : "uri"
},
{
"name" : "User name",
"identifier" : "u"
},
{
"name" : "Project name",
"identifier" : "p"
}
{
"name" : "Project branch",
"identifier" : "b"
}
public $parameters = array( array(
'uri'=>array(
'name'=>'Base URI',
'defaultValue'=>'https://gitlab.com'
),
'u'=>array(
'name'=>'User name',
'required'=>true
),
'p'=>array(
'name'=>'Project name',
'required'=>true
),
'b'=>array(
'name'=>'Project branch',
'defaultValue'=>'master'
)
));
]';
}
public function collectData(array $param){
$uri = $param['uri'].'/'.$param['u'].'/'.$param['p'].'/commits/';
if(isset($param['b'])){
$uri.=$param['b'];
}else{
$uri.='master';
}
$html = file_get_html($uri)
or $this->returnError('No results for Gitlab Commits of project '.$param['uri'].'/'.$param['u'].'/'.$param['p'], 404);
public function collectData(){
$uri = $this->getInput('uri').'/'.$this->getInput('u').'/'
.$this->getInput('p').'/commits/'.$this->getInput('b');
$html = $this->getSimpleHTMLDOM($uri)
or $this->returnServerError('No results for Gitlab Commits of project '.$uri);
foreach($html->find('li.commit') as $commit){
$item = new \Item();
$item->uri=$param['uri'];
$item = array();
$item['uri']=$this->getInput('uri');
foreach($commit->getElementsByTagName('a') as $a){
$classes=explode(' ',$a->getAttribute("class"));
if(in_array('commit-short-id',$classes) ||
in_array('commit_short_id',$classes)){
$href=$a->getAttribute('href');
$item->uri.=substr($href,strpos($href,'/'.$param['u'].'/'.$param['p']));
$item['uri'].=substr($href,strpos($href,'/'.$this->getInput('u').'/'.$this->getInput('p')));
}
if(in_array('commit-row-message',$classes)){
$item->title=$a->plaintext;
$item['title']=$a->plaintext;
}
if(in_array('commit-author-link',$classes)){
$item->name=trim($a->plaintext);
$item['author']=trim($a->plaintext);
}
}
$pre=$commit->find('pre',0);
if($pre){
$item->content=$pre->outertext;
$item['content']=$pre->outertext;
}else{
$item->content='';
$item['content']='';
}
$item->timestamp=strtotime($commit->find('time',0)->getAttribute('datetime'));
$item['timestamp']=strtotime($commit->find('time',0)->getAttribute('datetime'));
$this->items[]=$item;
}
}
public function getName(){
return 'Gitlab Commits';
}
public function getURI(){
return '';
}
public function getCacheDuration(){
return 3600; // one hour
}
}

41
bridges/GizmodoFRBridge.php
View File

@ -1,20 +1,18 @@
<?php
class GizmodoFRBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "polopollo";
public $name = "GizmodoFR";
public $uri = "http://www.gizmodo.fr/";
public $description = "Returns the 15 newest posts from GizmodoFR (full text).";
$this->maintainer = "polopollo";
$this->name = "GizmodoFR";
$this->uri = "http://www.gizmodo.fr/";
$this->description = "Returns the 15 newest posts from GizmodoFR (full text).";
$this->update = "2014-07-14";
}
public function collectData(array $param){
public function collectData(){
function GizmodoFRExtractContent($url) {
$articleHTMLContent = $this->file_get_html($url);
$articleHTMLContent = $this->getSimpleHTMLDOM($url);
if(!$articleHTMLContent){
return 'Could not load '.$url;
}
$text = $articleHTMLContent->find('div.entry-thumbnail', 0)->innertext;
$text = $text.$articleHTMLContent->find('div.entry-excerpt', 0)->innertext;
$text = $text.$articleHTMLContent->find('div.entry-content', 0)->innertext;
@ -26,16 +24,17 @@ class GizmodoFRBridge extends BridgeAbstract{
return $text;
}
$rssFeed = $this->file_get_html('http://www.gizmodo.fr/feed') or $this->returnError('Could not request http://www.gizmodo.fr/feed', 404);
$rssFeed = $this->getSimpleHTMLDOM($this->uri.'/feed')
or $this->returnServerError('Could not request '.$this->uri.'/feed');
$limit = 0;
foreach($rssFeed->find('item') as $element) {
if($limit < 15) {
$item = new \Item();
$item->title = $element->find('title', 0)->innertext;
$item->uri = $element->find('guid', 0)->plaintext;
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = GizmodoFRExtractContent($item->uri);
$item = array();
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = $element->find('guid', 0)->plaintext;
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = GizmodoFRExtractContent($item['uri']);
$this->items[] = $item;
$limit++;
}
@ -43,14 +42,6 @@ class GizmodoFRBridge extends BridgeAbstract{
}
public function getName(){
return 'GizmodoFR';
}
public function getURI(){
return 'http://www.gizmodo.fr/';
}
public function getCacheDuration(){
return 1800; // 30min
}

82
bridges/GooglePlusPostBridge.php
View File

@ -4,42 +4,28 @@ class GooglePlusPostBridge extends BridgeAbstract
protected $_title;
protected $_url;
public function loadMetadatas() {
public $maintainer = "Grummfy";
public $name = "Google Plus Post Bridge";
public $uri = "https://plus.google.com/";
public $description = "Returns user public post (without API).";
$this->maintainer = "Grummfy";
$this->name = "Google Plus Post Bridge";
$this->uri = "http://plus.google.com/";
$this->description = "Returns user public post (without API).";
$this->update = "2014-07-20";
public $parameters = array( array(
'username'=>array(
'name'=>'username or Id',
'required'=>true
)
));
$this->parameters[] =
'[
{
"name" : "username or Id",
"identifier" : "username"
}
]';
}
const GOOGLE_PLUS_BASE_URL = 'https://plus.google.com/';
public function collectData(array $param)
public function collectData()
{
if (!isset($param['username']))
{
$this->returnError('You must specify a username (?username=...).', 400);
}
$this->request = $param['username'];
// get content parsed
// $html = $this->file_get_html(__DIR__ . '/../posts2.html'
$html = $this->file_get_html(self::GOOGLE_PLUS_BASE_URL . urlencode($this->request) . '/posts'
// $html = $this->getSimpleHTMLDOM(__DIR__ . '/../posts2.html'
$html = $this->getSimpleHTMLDOM($this->uri . urlencode($this->getInput('username')) . '/posts'
// force language
, false, stream_context_create(array('http'=> array(
'header' => 'Accept-Language: fr,fr-be,fr-fr;q=0.8,en;q=0.4,en-us;q=0.2;*' . "\r\n"
)))
) OR $this->returnError('No results for this query.', 404);
) OR $this->returnServerError('No results for this query.');
// get title, url, ... there is a lot of intresting stuff in meta
$this->_title = $html->find('meta[property]', 0)->getAttribute('content');
@ -47,44 +33,44 @@ class GooglePlusPostBridge extends BridgeAbstract
// foreach ($html->find('meta') as $e)
// {
// $item = new \Item();
// $item->content = var_export($e->attr, true);
// $item = array();
// $item['content'] = var_export($e->attr, true);
// $this->items[] = $item;
// }
// div[jsmodel=XNmfOc]
foreach($html->find('div.yt') as $post)
{
$item = new \Item();
// $item->content = $post->find('div.Al', 0)->innertext;
$item->username = $item->fullname = $post->find('header.lea h3 a', 0)->innertext;
$item->id = $post->getAttribute('id');
// $item->title = $item->fullname = $post->find('header.lea', 0)->plaintext;
$item->avatar = $post->find('div.ys img', 0)->src;
$item = array();
// $item['content'] = $post->find('div.Al', 0)->innertext;
$item['username'] = $item['fullname'] = $post->find('header.lea h3 a', 0)->innertext;
$item['id'] = $post->getAttribute('id');
// $item['title'] = $item['fullname'] = $post->find('header.lea', 0)->plaintext;
$item['avatar'] = $post->find('div.ys img', 0)->src;
// var_dump((($post->find('a.o-U-s', 0)->getAllAttributes())));
$item->uri = self::GOOGLE_PLUS_BASE_URL . $post->find('a.o-U-s', 0)->href;
$item->timestamp = strtotime($post->find('a.o-U-s', 0)->plaintext);
$item['uri'] = $this->uri . $post->find('a.o-U-s', 0)->href;
$item['timestamp'] = strtotime($post->find('a.o-U-s', 0)->plaintext);
$this->items[] = $item;
// hashtag to treat : https://plus.google.com/explore/tag
$hashtags = array();
foreach($post->find('a.d-s') as $hashtag)
{
$hashtags[ trim($hashtag->plaintext) ] = self::GOOGLE_PLUS_BASE_URL . $hashtag->href;
$hashtags[ trim($hashtag->plaintext) ] = $this->uri . $hashtag->href;
}
$item->content = '';
$item['content'] = '';
// avatar display
$item->content .= '<div style="float:left; margin: 0 0.5em 0.5em 0;"><a href="' . self::GOOGLE_PLUS_BASE_URL . urlencode($this->request);
$item->content .= '"><img align="top" alt="avatar" src="' . $item->avatar.'" />' . $item->username . '</a></div>';
$item['content'] .= '<div style="float:left; margin: 0 0.5em 0.5em 0;"><a href="' . $this->uri . urlencode($this->getInput('username'));
$item['content'] .= '"><img align="top" alt="avatar" src="' . $item['avatar'].'" />' . $item['username'] . '</a></div>';
$content = $post->find('div.Al', 0);
// alter link
// $content = $content->innertext;
// $content = str_replace('href="./', 'href="' . self::GOOGLE_PLUS_BASE_URL, $content);
// $content = str_replace('href="photos', 'href="' . self::GOOGLE_PLUS_BASE_URL . 'photos', $content);
// $content = str_replace('href="./', 'href="' . $this->uri, $content);
// $content = str_replace('href="photos', 'href="' . $this->uri . 'photos', $content);
// XXX ugly but I don't have any idea how to do a better stuff, str_replace on link doesn't work as expected and ask too many checks
foreach($content->find('a') as $link)
{
@ -100,15 +86,15 @@ class GooglePlusPostBridge extends BridgeAbstract
{
$link->href = substr($link->href, 1);
}
$link->href = self::GOOGLE_PLUS_BASE_URL . $link->href;
$link->href = $this->uri . $link->href;
}
}
$content = $content->innertext;
$item->content .= '<div style="margin-top: -1.5em">' . $content . '</div>';
$item['content'] .= '<div style="margin-top: -1.5em">' . $content . '</div>';
// extract plaintext
$item->content_simple = $post->find('div.Al', 0)->plaintext;
$item['content_simple'] = $post->find('div.Al', 0)->plaintext;
}
// $html->save(__DIR__ . '/../posts2.html');
@ -121,7 +107,7 @@ class GooglePlusPostBridge extends BridgeAbstract
public function getURI()
{
return $this->_url ?: 'http://plus.google.com/';
return $this->_url ?: $this->uri;
}
public function getCacheDuration()

59
bridges/GoogleSearchBridge.php
View File

@ -9,61 +9,46 @@
*/
class GoogleSearchBridge extends BridgeAbstract{
private $request;
public $maintainer = "sebsauvage";
public $name = "Google search";
public $uri = "https://www.google.com/";
public $description = "Returns most recent results from Google search.";
public function loadMetadatas() {
$this->maintainer = "sebsauvage";
$this->name = "Google search";
$this->uri = "https://www.google.com/";
$this->description = "Returns most recent results from Google search.";
$this->update = "2014-05-25";
$this->parameters[] =
'[
{
"name" : "keyword",
"identifier" : "q"
}
]';
}
public $parameters = array( array(
'q'=>array(
'name'=>"keyword",
'required'=>true
)
));
public function collectData(array $param){
public function collectData(){
$html = '';
if (isset($param['q'])) { /* keyword search mode */
$this->request = $param['q'];
$html = $this->file_get_html('https://www.google.com/search?q=' . urlencode($this->request) . '&num=100&complete=0&tbs=qdr:y,sbd:1') or $this->returnError('No results for this query.', 404);
}
else{
$this->returnError('You must specify a keyword (?q=...).', 400);
}
$html = $this->getSimpleHTMLDOM($this->uri
.'search?q=' . urlencode($this->getInput('q'))
.'&num=100&complete=0&tbs=qdr:y,sbd:1')
or $this->returnServerError('No results for this query.');
$emIsRes = $html->find('div[id=ires]',0);
if( !is_null($emIsRes) ){
foreach($emIsRes->find('li[class=g]') as $element) {
$item = new Item();
$item = array();
// Extract direct URL from google href (eg. /url?q=...)
$t = $element->find('a[href]',0)->href;
$item->uri = ''.$t;
$item['uri'] = ''.$t;
parse_str(parse_url($t, PHP_URL_QUERY),$parameters);
if (isset($parameters['q'])) { $item->uri = $parameters['q']; }
$item->title = $element->find('h3',0)->plaintext;
$item->content = $element->find('span[class=st]',0)->plaintext;
if (isset($parameters['q'])) { $item['uri'] = $parameters['q']; }
$item['title'] = $element->find('h3',0)->plaintext;
$item['content'] = $element->find('span[class=st]',0)->plaintext;
$this->items[] = $item;
}
}
}
public function getName(){
return (!empty($this->request) ? $this->request .' - ' : '') .'Google search';
}
public function getURI(){
return 'http://google.com';
return $this->getInput('q') .' - Google search';
}
public function getCacheDuration(){

68
bridges/GuruMedBridge.php
View File

@ -1,54 +1,32 @@
<?php
class GuruMedBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "qwertygc";
$this->name = "GuruMed";
$this->uri = "http://www.gurumed.org";
$this->description = "Returns the 5 newest posts from Gurumed (full text)";
$this->update = "03/10/2015";
public $maintainer = "qwertygc";
public $name = "GuruMed";
public $uri = "http://www.gurumed.org";
public $description = "Returns the 5 newest posts from Gurumed (full text)";
private function GurumedStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
public function collectData(array $param){
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri.'feed')
or $this->returnServerError('Could not request Gurumed.');
$limit = 0;
function GurumedStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function GurumedExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = $html2->find('div.entry', 0)->innertext;
return $text;
}
$html = $this->file_get_html('http://gurumed.org/feed') or $this->returnError('Could not request Gurumed.', 404);
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 5) {
$item = new \Item();
$item->title = GurumedStripCDATA($element->find('title', 0)->innertext);
$item->uri = GurumedStripCDATA($element->find('guid', 0)->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = GurumedExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
foreach($html->find('item') as $element) {
if($limit < 5) {
$item = array();
$item['title'] = $this->GurumedStripCDATA($element->find('title', 0)->innertext);
$item['uri'] = $this->GurumedStripCDATA($element->find('guid', 0)->plaintext);
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $this->GurumedStripCDATA(strip_tags($element->find('description', 0), '<p><a><br>'));
$this->items[] = $item;
$limit++;
}
}
}
}
public function getName(){
return 'Gurumed';
}
public function getURI(){
return 'http://gurumed.org/';
}
public function getCacheDuration(){
return 3600; // 1 hour
}
}

75
bridges/HDWallpapersBridge.php
View File

@ -1,54 +1,36 @@
<?php
class HDWallpapersBridge extends BridgeAbstract {
public $maintainer = "nel50n";
public $name = "HD Wallpapers Bridge";
public $uri = "http://www.hdwallpapers.in/";
public $description = "Returns the latests wallpapers from HDWallpapers";
private $category;
private $resolution;
public function loadMetadatas() {
$this->maintainer = "nel50n";
$this->name = "HD Wallpapers Bridge";
$this->uri = "http://www.hdwallpapers.in/";
$this->description = "Returns the latests wallpapers from HDWallpapers";
$this->update = "2015-04-08";
$this->parameters[] =
'[
{
"name" : "category",
"identifier" : "c"
},
{
"name" : "max number of wallpapers",
"identifier" : "m"
},
{
"name" : "resolution",
"identifier" : "r",
"exampleValue" : "1920x1200, 1680x1050, ..."
}
]';
}
public function collectData(array $param){
$html = '';
$baseUri = 'http://www.hdwallpapers.in';
$this->category = $param['c'] ?: 'latest_wallpapers'; // Latest default
$this->resolution = $param['r'] ?: '1920x1200'; // Wide wallpaper default
public $parameters = array( array(
'c'=>array(
'name'=>'category',
'defaultValue'=>'latest_wallpapers'
),
'm'=>array('name'=>'max number of wallpapers'),
'r'=>array(
'name'=>'resolution',
'defaultValue'=>'1920x1200',
'exampleValue'=>'1920x1200, 1680x1050,…'
)
));
public function collectData(){
$category = $this->category;
if (strrpos($category, 'wallpapers') !== strlen($category)-strlen('wallpapers')) {
$category .= '-desktop-wallpapers';
}
$num = 0;
$max = $param['m'] ?: 14;
$max = $this->getInput('m') ?: 14;
$lastpage = 1;
for ($page = 1; $page <= $lastpage; $page++) {
$link = $baseUri.'/'.$category.'/page/'.$page;
$html = $this->file_get_html($link) or $this->returnError('No results for this query.', 404);
$link = $this->uri.'/'.$category.'/page/'.$page;
$html = $this->getSimpleHTMLDOM($link) or $this->returnServerError('No results for this query.');
if ($page === 1) {
preg_match('/page\/(\d+)$/', $html->find('.pagination a', -2)->href, $matches);
@ -58,13 +40,12 @@ class HDWallpapersBridge extends BridgeAbstract {
foreach($html->find('.wallpapers .wall a') as $element) {
$thumbnail = $element->find('img', 0);
$item = new \Item();
$item = array();
// http://www.hdwallpapers.in/download/yosemite_reflections-1680x1050.jpg
$item->uri = $baseUri.'/download'.str_replace('wallpapers.html', $this->resolution.'.jpg', $element->href);
$item->timestamp = time();
$item->title = $element->find('p', 0)->text();
$item->thumbnailUri = $baseUri.$thumbnail->src;
$item->content = $item->title.'<br><a href="'.$item->uri.'"><img src="'.$item->thumbnailUri.'" /></a>';
$item['uri'] = $this->uri.'/download'.str_replace('wallpapers.html', $this->getInput('r').'.jpg', $element->href);
$item['timestamp'] = time();
$item['title'] = $element->find('p', 0)->text();
$item['content'] = $item['title'].'<br><a href="'.$item['uri'].'"><img src="'.$this->uri.$thumbnail->src.'" /></a>';
$this->items[] = $item;
$num++;
@ -75,11 +56,7 @@ class HDWallpapersBridge extends BridgeAbstract {
}
public function getName(){
return 'HDWallpapers - '.str_replace(['__', '_'], [' & ', ' '], $this->category).' ['.$this->resolution.']';
}
public function getURI(){
return 'http://www.hdwallpapers.in';
return 'HDWallpapers - '.str_replace(['__', '_'], [' & ', ' '], $this->getInput('c')).' ['.$this->getInput('r').']';
}
public function getCacheDuration(){

40
bridges/HentaiHavenBridge.php
View File

@ -1,37 +1,25 @@
<?php
class HentaiHavenBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "albirew";
public $name = "Hentai Haven";
public $uri = "http://hentaihaven.org/";
public $description = "Returns releases from Hentai Haven";
$this->maintainer = "albirew";
$this->name = "Hentai Haven";
$this->uri = "http://hentaihaven.org/";
$this->description = "Returns releases from Hentai Haven";
$this->update = "2015-01-16";
}
public function collectData(array $param){
$html = $this->file_get_html('http://hentaihaven.org/') or $this->returnError('Could not request Hentai Haven.', 404);
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Could not request Hentai Haven.');
foreach($html->find('div.zoe-grid') as $element) {
$item = new \Item();
$item->uri = $this->getURI().$element->find('div.brick-content h3 a', 0)->href;
$item->thumbnailUri = $element->find('a.thumbnail-image img', 0)->getAttribute('data-src');
$item->title = mb_convert_encoding(trim($element->find('div.brick-content h3 a', 0)->innertext), 'UTF-8', 'HTML-ENTITIES');
$item->tags = $element->find('div.oFlyout_bg div.oFlyout div.flyoutContent span.tags', 0)->plaintext;
$item->content = 'Tags: ' . $item->tags.'<br><br><a href="' . $item->uri . '"><img width="300" height="169" src="' . $item->thumbnailUri . '" /></a><br>' . $element->find('div.oFlyout_bg div.oFlyout div.flyoutContent p.description', 0)->innertext;
$item = array();
$item['uri'] = $element->find('div.brick-content h3 a', 0)->href;
$thumbnailUri = $element->find('a.thumbnail-image img', 0)->getAttribute('data-src');
$item['title'] = mb_convert_encoding(trim($element->find('div.brick-content h3 a', 0)->innertext), 'UTF-8', 'HTML-ENTITIES');
$item['tags'] = $element->find('div.oFlyout_bg div.oFlyout div.flyoutContent span.tags', 0)->plaintext;
$item['content'] = 'Tags: ' . $item['tags'].'<br><br><a href="' . $item['uri'] . '"><img width="300" height="169" src="' . $thumbnailUri . '" /></a><br>' . $element->find('div.oFlyout_bg div.oFlyout div.flyoutContent p.description', 0)->innertext;
$this->items[] = $item;
}
}
public function getName(){
return 'Hentai Haven';
}
public function getURI(){
return '';
}
public function getDescription(){
return "Hentai Haven | Everything Else is Irrelevant";
}
public function getCacheDuration(){
return 21600; // 6 hours
}

54
bridges/IdenticaBridge.php
View File

@ -1,52 +1,38 @@
<?php
class IdenticaBridge extends BridgeAbstract{
private $request;
public $maintainer = "mitsukarenai";
public $name = "Identica Bridge";
public $uri = "https://identi.ca/";
public $description = "Returns user timelines";
public function loadMetadatas() {
public $parameters = array( array(
'u'=>array(
'name'=>'username',
'required'=>true
)
));
$this->maintainer = "mitsukarenai";
$this->name = "Identica Bridge";
$this->uri = "https://identi.ca/";
$this->description = "Returns user timelines";
$this->update = "2014-05-25";
$this->parameters[] =
'[
{
"name" : "username",
"identifier" : "u"
}
]';
}
public function collectData(array $param){
$html = '';
if (isset($param['u'])) { /* user timeline mode */
$this->request = $param['u'];
$html = $this->file_get_html('https://identi.ca/'.urlencode($this->request)) or $this->returnError('Requested username can\'t be found.', 404);
}
else {
$this->returnError('You must specify an Identica username (?u=...).', 400);
}
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->getURI())
or $this->returnServerError('Requested username can\'t be found.');
foreach($html->find('li.major') as $dent) {
$item = new \Item();
$item->uri = html_entity_decode($dent->find('a', 0)->href); // get dent link
$item->timestamp = strtotime($dent->find('abbr.easydate', 0)->plaintext); // extract dent timestamp
$item->content = trim($dent->find('div.activity-content', 0)->innertext); // extract dent text
$item->title = $param['u'] . ' | ' . $item->content;
$item = array();
$item['uri'] = html_entity_decode($dent->find('a', 0)->href); // get dent link
$item['timestamp'] = strtotime($dent->find('abbr.easydate', 0)->plaintext); // extract dent timestamp
$item['content'] = trim($dent->find('div.activity-content', 0)->innertext); // extract dent text
$item['title'] = $this->getInput('u') . ' | ' . $item['content'];
$this->items[] = $item;
}
}
public function getName(){
return (!empty($this->request) ? $this->request .' - ' : '') .'Identica Bridge';
return $this->getInput('u') .' - Identica Bridge';
}
public function getURI(){
return 'https://identica.com';
return $this->uri.urlencode($this->getInput('u'));
}
public function getCacheDuration(){

75
bridges/InstagramBridge.php
View File

@ -1,88 +1,71 @@
<?php
class InstagramBridge extends BridgeAbstract{
private $request;
public $maintainer = "pauder";
public $name = "Instagram Bridge";
public $uri = "http://instagram.com/";
public $description = "Returns the newest images";
public function loadMetadatas() {
public $parameters = array( array(
'u'=>array(
'name'=>'username',
'required'=>true
)
));
$this->maintainer = "pauder";
$this->name = "Instagram Bridge";
$this->uri = "http://instagram.com/";
$this->description = "Returns the newest images";
$this->update = "2014-05-25";
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->getURI())
or $this->returnServerError('Could not request Instagram.');
$this->parameters[] =
'[
{
"name" : "username",
"identifier" : "u"
}
]';
}
public function collectData(array $param){
$html = '';
if (isset($param['u'])) { /* user timeline mode */
$this->request = $param['u'];
$html = $this->file_get_html('http://instagram.com/'.urlencode($this->request)) or $this->returnError('Could not request Instagram.', 404);
}
else {
$this->returnError('You must specify a Instagram username (?u=...).', 400);
}
$innertext = null;
foreach($html->find('script') as $script)
{
if ('' === $script->innertext) {
continue;
}
$pos = strpos(trim($script->innertext), 'window._sharedData');
if (0 !== $pos)
{
continue;
}
$innertext = $script->innertext;
break;
}
$json = trim(substr($innertext, $pos+18), ' =;');
$data = json_decode($json);
$userMedia = $data->entry_data->ProfilePage[0]->user->media->nodes;
foreach($userMedia as $media)
{
$item = new \Item();
$item->uri = "https://instagram.com/p/".$media->code."/";
$item->content = '<img src="' . htmlentities($media->display_src) . '" />';
$item = array();
$item['uri'] = $this->uri.'/p/'.$media->code.'/';
$item['content'] = '<img src="' . htmlentities($media->display_src) . '" />';
if (isset($media->caption))
{
$item->title = $media->caption;
$item['title'] = $media->caption;
} else {
$item->title = basename($media->display_src);
$item['title'] = basename($media->display_src);
}
$item->timestamp = $media->date;
$item['timestamp'] = $media->date;
$this->items[] = $item;
}
}
public function getName(){
return (!empty($this->request) ? $this->request .' - ' : '') .'Instagram Bridge';
return $this->getInput('u') .' - Instagram Bridge';
}
public function getURI(){
return 'http://instagram.com/';
}
public function getCacheDuration(){
return 3600;
return $this->uri.urlencode($this->getInput('u'));
}
}

471
bridges/IsoHuntBridge.php Normal file
View File

@ -0,0 +1,471 @@
<?php
class IsoHuntBridge extends BridgeAbstract{
public $maintainer = 'logmanoriginal';
public $name = 'isoHunt Bridge';
public $uri = 'https://isohunt.to/';
public $description = 'Returns the latest results by category or search result';
public $parameters = array(
/*
* Get feeds for one of the "latest" categories
* Notice: The categories "News" and "Top Searches" are received from the main page
* Elements are sorted by name ascending!
*/
'By "Latest" category' => array(
'latest_category'=>array(
'name'=>'Latest category',
'type'=>'list',
'required'=>true,
'title'=>'Select your category',
'defaultValue'=>'news',
'values'=>array(
'Hot Torrents'=>'hot_torrents',
'News'=>'news',
'Releases'=>'releases',
'Torrents'=>'torrents'
)
)
),
/*
* Get feeds for one of the "torrent" categories
* Make sure to add new categories also to get_torrent_category_index($)!
* Elements are sorted by name ascending!
*/
'By "Torrent" category' => array(
'torrent_category'=>array(
'name'=>'Torrent category',
'type'=>'list',
'required'=>true,
'title'=>'Select your category',
'defaultValue'=>'anime',
'values'=>array(
'Adult'=>'adult',
'Anime'=>'anime',
'Books'=>'books',
'Games'=>'games',
'Movies'=>'movies',
'Music'=>'music',
'Other'=>'other',
'Series & TV'=>'series_tv',
'Software'=>'software'
)
),
'torrent_popularity'=>array(
'name'=>'Sort by popularity',
'type'=>'checkbox',
'title'=>'Activate to receive results by popularity'
)
),
/*
* Get feeds for a specific search request
*/
'Search torrent by name' => array(
'search_name'=>array(
'name'=>'Name',
'required'=>true,
'title'=>'Insert your search query',
'exampleValue'=>'Bridge'
),
'search_category'=>array(
'name'=>'Category',
'type'=>'list',
'title'=>'Select your category',
'defaultValue'=>'all',
'values'=>array(
'Adult'=>'adult',
'All'=>'all',
'Anime'=>'anime',
'Books'=>'books',
'Games'=>'games',
'Movies'=>'movies',
'Music'=>'music',
'Other'=>'other',
'Series & TV'=>'series_tv',
'Software'=>'software'
)
)
)
);
public function getURI(){
$uri=$this->uri;
switch($this->queriedContext){
case 'By "Latest" category':
switch($this->getInput('latest_category')){
case 'hot_torrents':
$uri .= 'statistic/hot/torrents';
break;
case 'news':
break;
case 'releases':
$uri .= 'releases.php';
break;
case 'torrents':
$uri .= 'latest.php';
break;
}
break;
case 'By "Torrent" category':
$uri .= $this->build_category_uri(
$this->getInput('torrent_category'),
$this->getInput('torrent_popularity')
);
break;
case 'Search torrent by name':
$category=$this->getInput('search_category');
$uri .= $this->build_category_uri($category);
if($category!=='movies')
$uri .= '&ihq=' . urlencode($this->getInput('search_name'));
break;
}
return $uri;
}
public function getName(){
switch($this->queriedContext){
case 'By "Latest" category':
$categoryName =
array_search(
$this->getInput('latest_category'),
$this->parameters['By "Latest" category']['latest_category']['values']
);
$name = 'Latest '.$categoryName.' - ' . $this->name;
break;
case 'By "Torrent" category':
$categoryName =
array_search(
$this->getInput('torrent_category'),
$this->parameters['By "Torrent" category']['torrent_category']['values']
);
$name = 'Category: ' . $categoryName . ' - ' . $this->name;
break;
case 'Search torrent by name':
$categoryName =
array_search(
$this->getInput('search_category'),
$this->parameters['Search torrent by name']['search_category']['values']
);
$name = 'Search: "' . $this->getInput('search_name') . '" in category: ' . $categoryName . ' - ' . $this->name;
break;
}
return $name;
}
public function collectData(){
$html = $this->load_html($this->getURI());
switch($this->queriedContext){
case 'By "Latest" category':
switch($this->getInput('latest_category')){
case 'hot_torrents':
$this->get_latest_hot_torrents($html);
break;
case 'news':
$this->get_latest_news($html);
break;
case 'releases':
case 'torrents':
$this->get_latest_torrents($html);
break;
}
break;
case 'By "Torrent" category':
if($this->getInput('torrent_category') === 'movies'){
// This one is special (content wise)
$this->get_movie_torrents($html);
}else{
$this->get_latest_torrents($html);
}
break;
case 'Search torrent by name':
if( $this->getInput('search_category') === 'movies'){
// This one is special (content wise)
$this->get_movie_torrents($html);
} else {
$this->get_latest_torrents($html);
}
break;
}
}
public function getCacheDuration(){
return 300; // 5 minutes
}
#region Helper functions for "Movie Torrents"
private function get_movie_torrents($html){
$container = $html->find('div#w0', 0);
if(!$container)
$this->returnServerError('Unable to find torrent container!');
$torrents = $container->find('article');
if(!$torrents)
$this->returnServerError('Unable to find torrents!');
foreach($torrents as $torrent){
$anchor = $torrent->find('a', 0);
if(!$anchor)
$this->returnServerError('Unable to find anchor!');
$date = $torrent->find('small', 0);
if(!$date)
$this->returnServerError('Unable to find date!');
$item = array();
$item['uri'] = $this->fix_relative_uri($anchor->href);
$item['title'] = $anchor->title;
// $item['author'] =
$item['timestamp'] = strtotime($date->plaintext);
$item['content'] = $this->fix_relative_uri($torrent->innertext);
$this->items[] = $item;
}
}
#endregion
#region Helper functions for "Latest Hot Torrents"
private function get_latest_hot_torrents($html){
$container = $html->find('div#serps', 0);
if(!$container)
$this->returnServerError('Unable to find torrent container!');
$torrents = $container->find('tr');
if(!$torrents)
$this->returnServerError('Unable to find torrents!');
// Remove first element (header row)
$torrents = array_slice($torrents, 1);
foreach($torrents as $torrent){
$cell = $torrent->find('td', 0);
if(!$cell)
$this->returnServerError('Unable to find cell!');
$element = $cell->find('a', 0);
if(!$element)
$this->returnServerError('Unable to find element!');
$item = array();
$item['uri'] = $element->href;
$item['title'] = $element->plaintext;
// $item['author'] =
// $item['timestamp'] =
// $item['content'] =
$this->items[] = $item;
}
}
#endregion
#region Helper functions for "Latest News"
private function get_latest_news($html){
$container = $html->find('div#postcontainer', 0);
if(!$container)
$this->returnServerError('Unable to find post container!');
$posts = $container->find('div.index-post');
if(!$posts)
$this->returnServerError('Unable to find posts!');
foreach($posts as $post){
$item = array();
$item['uri'] = $this->latest_news_extract_uri($post);
$item['title'] = $this->latest_news_extract_title($post);
$item['author'] = $this->latest_news_extract_author($post);
$item['timestamp'] = $this->latest_news_extract_timestamp($post);
$item['content'] = $this->latest_news_extract_content($post);
$this->items[] = $item;
}
}
private function latest_news_extract_author($post){
$author = $post->find('small', 0);
if(!$author)
$this->returnServerError('Unable to find author!');
// The author is hidden within a string like: 'Posted by {author} on {date}'
preg_match('/Posted\sby\s(.*)\son/i', $author->innertext, $matches);
return $matches[1];
}
private function latest_news_extract_timestamp($post){
$date = $post->find('small', 0);
if(!$date)
$this->returnServerError('Unable to find date!');
// The date is hidden within a string like: 'Posted by {author} on {date}'
preg_match('/Posted\sby\s.*\son\s(.*)/i', $date->innertext, $matches);
$timestamp = strtotime($matches[1]);
// Make sure date is not in the future (dates are given like 'Nov. 20' without year)
if($timestamp > time()){
$timestamp = strtotime('-1 year', $timestamp);
}
return $timestamp;
}
private function latest_news_extract_title($post){
$title = $post->find('a', 0);
if(!$title)
$this->returnServerError('Unable to find title!');
return $title->plaintext;
}
private function latest_news_extract_uri($post){
$uri = $post->find('a', 0);
if(!$uri)
$this->returnServerError('Unable to find uri!');
return $uri->href;
}
private function latest_news_extract_content($post){
$content = $post->find('div', 0);
if(!$content)
$this->returnServerError('Unable to find content!');
// Remove <h2>...</h2> (title)
foreach($content->find('h2') as $element){
$element->outertext = '';
}
// Remove <small>...</small> (author)
foreach($content->find('small') as $element){
$element->outertext = '';
}
return $content->innertext;
}
#endregion
#region Helper functions for "Latest Torrents", "Latest Releases" and "Torrent Category"
private function get_latest_torrents($html){
$container = $html->find('div#serps', 0);
if(!$container)
$this->returnServerError('Unable to find torrent container!');
$torrents = $container->find('tr[data-key]');
if(!$torrents)
$this->returnServerError('Unable to find torrents!');
foreach($torrents as $torrent){
$item = array();
$item['uri'] = $this->latest_torrents_extract_uri($torrent);
$item['title'] = $this->latest_torrents_extract_title($torrent);
$item['author'] = $this->latest_torrents_extract_author($torrent);
$item['timestamp'] = $this->latest_torrents_extract_timestamp($torrent);
$item['content'] = ''; // There is no valuable content
$this->items[] = $item;
}
}
private function latest_torrents_extract_title($torrent){
$cell = $torrent->find('td.title-row', 0);
if(!$cell)
$this->returnServerError('Unable to find title cell!');
$title = $cell->find('span', 0);
if(!$title)
$this->returnServerError('Unable to find title!');
return $title->plaintext;
}
private function latest_torrents_extract_uri($torrent){
$cell = $torrent->find('td.title-row', 0);
if(!$cell)
$this->returnServerError('Unable to find title cell!');
$uri = $cell->find('a', 0);
if(!$uri)
$this->returnServerError('Unable to find uri!');
return $this->fix_relative_uri($uri->href);
}
private function latest_torrents_extract_author($torrent){
$cell = $torrent->find('td.user-row', 0);
if(!$cell)
return; // No author
$user = $cell->find('a', 0);
if(!$user)
$this->returnServerError('Unable to find user!');
return $user->plaintext;
}
private function latest_torrents_extract_timestamp($torrent){
$cell = $torrent->find('td.date-row', 0);
if(!$cell)
$this->returnServerError('Unable to find date cell!');
return strtotime('-' . $cell->plaintext, time());
}
#endregion
#region Generic helper functions
private function load_html($uri){
$html = $this->getSimpleHTMLDOM($uri);
if(!$html)
$this->returnServerError('Unable to load ' . $uri . '!');
return $html;
}
private function fix_relative_uri($uri){
return preg_replace('/\//i', $this->uri, $uri, 1);
}
private function build_category_uri($category, $order_popularity = false){
switch($category){
case 'anime': $index = 1; break;
case 'software' : $index = 2; break;
case 'games' : $index = 3; break;
case 'adult' : $index = 4; break;
case 'movies' : $index = 5; break;
case 'music' : $index = 6; break;
case 'other' : $index = 7; break;
case 'series_tv' : $index = 8; break;
case 'books': $index = 9; break;
case 'all':
default: $index = 0; break;
}
return 'torrents/?iht=' . $index . '&ihs=' . ($order_popularity ? 1 : 0) . '&age=0';
}
#endregion
}

91
bridges/JapanExpoBridge.php
View File

@ -1,34 +1,18 @@
<?php
class JapanExpoBridge extends BridgeAbstract{
class JapanExpoBridge extends HttpCachingBridgeAbstract {
public function loadMetadatas() {
$this->maintainer = 'Ginko';
$this->name = 'JapanExpo';
$this->uri = 'http://www.japan-expo-paris.com/fr/actualites';
$this->description = 'Returns most recent entries from Japan Expo actualités.';
$this->update = '2016-06-12';
$this->parameters[] =
'[
{
"name" : "Mode",
"type" : "list",
"identifier" : "mode",
"values" :
[
{
"name" : "Titles only",
"value" : "light"
},
{
"name" : "Full Contents",
"value" : "full"
}
]
}
]';
}
public $maintainer = 'Ginko';
public $name = 'Japan Expo Actualités';
public $uri = 'http://www.japan-expo-paris.com/fr/actualites';
public $description = 'Returns most recent entries from Japan Expo actualités.';
public $parameters = array( array(
'mode'=>array(
'name'=>'Show full contents',
'type'=>'checkbox',
)
));
public function collectData(array $param) {
public function collectData(){
function french_pubdate_to_timestamp($date_to_parse) {
return strtotime(
@ -58,9 +42,9 @@ class JapanExpoBridge extends BridgeAbstract{
}
};
$link = 'http://www.japan-expo-paris.com/fr/actualites';
$html = $this->file_get_html($link) or $this->returnError('Could not request JapanExpo: '.$link , 500);
$fullcontent = (!empty($param['mode']) && $param['mode'] == 'full');
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Could not request JapanExpo: '.$this->uri);
$fullcontent = $this->getInput('mode');
$count = 0;
foreach ($html->find('a._tile2') as $element) {
@ -72,19 +56,21 @@ class JapanExpoBridge extends BridgeAbstract{
$thumbnail = trim($img_search_result[1], "'");
if ($fullcontent) {
if ($count < 5) {
$article_html = $this->file_get_html($url) or $this->returnError('Could not request JapanExpo: '.$url , 500);
$header = $article_html->find('header.pageHeadBox', 0);
$timestamp = strtotime($header->find('time', 0)->datetime);
$title_html = $header->find('div.section', 0)->next_sibling();
$title = $title_html->plaintext;
$headings = $title_html->next_sibling()->outertext;
$article = $article_html->find('div.content', 0)->innertext;
$article = preg_replace_callback('/<img [^>]+ style="[^\(]+\(\'([^\']+)\'[^>]+>/i', $convert_article_images, $article);
$content = $headings.$article;
} else {
break;
if ($count >= 5) {
break;
}
if($this->get_cached_time($url) <= strtotime('-24 hours'))
$this->remove_from_cache($url);
$article_html = $this->get_cached($url) or $this->returnServerError('Could not request JapanExpo: '.$url);
$header = $article_html->find('header.pageHeadBox', 0);
$timestamp = strtotime($header->find('time', 0)->datetime);
$title_html = $header->find('div.section', 0)->next_sibling();
$title = $title_html->plaintext;
$headings = $title_html->next_sibling()->outertext;
$article = $article_html->find('div.content', 0)->innertext;
$article = preg_replace_callback('/<img [^>]+ style="[^\(]+\(\'([^\']+)\'[^>]+>/i', $convert_article_images, $article);
$content = $headings.$article;
} else {
$date_text = $element->find('span.date', 0)->plaintext;
$timestamp = french_pubdate_to_timestamp($date_text);
@ -92,25 +78,16 @@ class JapanExpoBridge extends BridgeAbstract{
$content = '<img src="'.$thumbnail.'"></img><br />'.$date_text.'<br /><a href="'.$url.'">Lire l\'article</a>';
}
$item = new \Item();
$item->uri = $url;
$item->title = $title;
$item->timestamp = $timestamp;
$item->thumbnailUri = $thumbnail;
$item->content = $content;
$item = array();
$item['uri'] = $url;
$item['title'] = $title;
$item['timestamp'] = $timestamp;
$item['content'] = $content;
$this->items[] = $item;
$count++;
}
}
public function getName(){
return 'Japan Expo Actualités';
}
public function getURI(){
return 'http://www.japan-expo-paris.com/fr/actualites';
}
public function getCacheDuration(){
return 14400; // 4 hours
}

71
bridges/KonachanBridge.php
View File

@ -1,64 +1,45 @@
<?php
class KonachanBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "mitsukarenai";
public $name = "Konachan";
public $uri = "http://konachan.com/";
public $description = "Returns images from given page";
$this->maintainer = "mitsukarenai";
$this->name = "Konachan";
$this->uri = "http://konachan.com/";
$this->description = "Returns images from given page";
$this->update = "2014-05-25";
public $parameters = array( array(
'p'=>array(
'name'=>'page',
'defaultValue'=>1,
'type'=>'number'
),
't'=>array('name'=>'tags')
));
$this->parameters[] =
'[
{
"name" : "page",
"identifier" : "p",
"type" : "number"
},
{
"name" : "tags",
"identifier" : "t"
}
]';
}
public function collectData(){
$html = $this->getSimpleHTMLDOM(
$this->uri.'/post?'
.'&page='.$this->getInput('p')
.'&tags='.urlencode($this->getInput('t'))
) or $this->returnServerError('Could not request Konachan.');
public function collectData(array $param){
$page = 1;$tags='';
if (isset($param['p'])) {
$page = (int)preg_replace("/[^0-9]/",'', $param['p']);
}
if (isset($param['t'])) {
$tags = urlencode($param['t']);
}
$html = $this->file_get_html("http://konachan.com/post?page=$page&tags=$tags") or $this->returnError('Could not request Konachan.', 404);
$input_json = explode('Post.register(', $html);
foreach($input_json as $element)
$data[] = preg_replace('/}\)(.*)/', '}', $element);
unset($data[0]);
foreach($data as $datai) {
$json = json_decode($datai, TRUE);
$item = new \Item();
$item->uri = 'http://konachan.com/post/show/'.$json['id'];
$item->postid = $json['id'];
$item->timestamp = $json['created_at'];
$item->imageUri = $json['file_url'];
$item->thumbnailUri = $json['preview_url'];
$item->title = 'Konachan | '.$json['id'];
$item->content = '<a href="' . $item->imageUri . '"><img src="' . $item->thumbnailUri . '" /></a><br>Tags: '.$json['tags'];
$item = array();
$item['uri'] = $this->uri.'/post/show/'.$json['id'];
$item['postid'] = $json['id'];
$item['timestamp'] = $json['created_at'];
$item['imageUri'] = $json['file_url'];
$item['title'] = 'Konachan | '.$json['id'];
$item['content'] = '<a href="' . $item['imageUri'] . '"><img src="' . $json['preview_url'] . '" /></a><br>Tags: '.$json['tags'];
$this->items[] = $item;
}
}
public function getName(){
return 'Konachan';
}
public function getURI(){
return 'http://konachan.com/post';
}
public function getCacheDuration(){
return 1800; // 30 minutes
}

76
bridges/KoreusBridge.php
View File

@ -1,56 +1,38 @@
<?php
class KoreusBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "pit-fgfjiudghdf";
$this->name = "Koreus";
$this->uri = "http://www.koreus.com/";
$this->description = "Returns the 5 newest posts from Koreus (full text)";
$this->update = "2014-05-26";
public $maintainer = "pit-fgfjiudghdf";
public $name = "Koreus";
public $uri = "http://www.koreus.com/";
public $description = "Returns the 5 newest posts from Koreus (full text)";
private function KoreusStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
public function collectData(array $param){
private function KoreusExtractContent($url) {
$html2 = $this->getSimpleHTMLDOM($url);
$text = $html2->find('p[class=itemText]', 0)->innertext;
$text = utf8_encode(preg_replace('/(Sur le m.+?)+$/i','',$text));
return $text;
}
function KoreusStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function KoreusExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = $html2->find('p[class=itemText]', 0)->innertext;
$text = utf8_encode(preg_replace('/(Sur le m.+?)+$/i','',$text));
return $text;
}
$html = $this->file_get_html('http://feeds.feedburner.com/Koreus-articles') or $this->returnError('Could not request Koreus.', 404);
$limit = 0;
public function collectData(){
$html = $this->getSimpleHTMLDOM('http://feeds.feedburner.com/Koreus-articles') or $this->returnServerError('Could not request Koreus.');
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 5) {
$item = new \Item();
$item->title = KoreusStripCDATA($element->find('title', 0)->innertext);
$item->uri = KoreusStripCDATA($element->find('guid', 0)->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = KoreusExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'Koreus';
}
public function getURI(){
return 'http://www.koreus.com/';
}
public function getCacheDuration(){
return 3600; // 1 hour
}
foreach($html->find('item') as $element) {
if($limit < 5) {
$item = array();
$item['title'] = $this->KoreusStripCDATA($element->find('title', 0)->innertext);
$item['uri'] = $this->KoreusStripCDATA($element->find('guid', 0)->plaintext);
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $this->KoreusExtractContent($item['uri']);
$this->items[] = $item;
$limit++;
}
}
}
}

242
bridges/KununuBridge.php Normal file
View File

@ -0,0 +1,242 @@
<?php
class KununuBridge extends HttpCachingBridgeAbstract {
public $maintainer = "logmanoriginal";
public $name = "Kununu Bridge";
public $uri = "https://www.kununu.com/";
public $description = "Returns the latest reviews for a company and site of your choice.";
public $parameters = array(
'global' => array(
'site'=>array(
'name'=>'Site',
'type'=>'list',
'required'=>true,
'title'=>'Select your site',
'values'=>array(
'Austria'=>'at',
'Germany'=>'de',
'Switzerland'=>'ch',
'United States'=>'us'
)
),
'full'=>array(
'name'=>'Load full article',
'type'=>'checkbox',
'required'=>false,
'exampleValue'=>'checked',
'title'=>'Activate to load full article'
)
),
array(
'company'=>array(
'name'=>'Company',
'required'=>true,
'exampleValue'=>'kununu-us',
'title'=>'Insert company name (i.e. Kununu US) or URI path (i.e. kununu-us)'
)
)
);
private $companyName='';
public function getURI(){
$company = $this->encode_umlauts(strtolower(str_replace(' ', '-', trim($this->getInput('company')))));
$site=$this->getInput('site');
$section = '';
switch($site){
case 'at':
case 'de':
case 'ch':
$section = 'kommentare';
break;
case 'us':
$section = 'reviews';
break;
}
return $this->uri.$site.'/'.$company.'/'.$section;
}
function getName(){
$company = $this->encode_umlauts(strtolower(str_replace(' ', '-', trim($this->getInput('company')))));
return ($this->companyName?:$company).' - '.$this->name;
}
public function collectData(){
$full = $this->getInput('full');
// Load page
$html = $this->getSimpleHTMLDOM($this->getURI());
if(!$html)
$this->returnServerError('Unable to receive data from ' . $this->getURI() . '!');
// Update name for this request
$this->companyName = $this->extract_company_name($html);
// Find the section with all the panels (reviews)
$section = $html->find('section.kununu-scroll-element', 0);
if($section === false)
$this->returnServerError('Unable to find panel section!');
// Find all articles (within the panels)
$articles = $section->find('article');
if($articles === false || empty($articles))
$this->returnServerError('Unable to find articles!');
// Go through all articles
foreach($articles as $article){
$item = array();
$item['author'] = $this->extract_article_author_position($article);
$item['timestamp'] = $this->extract_article_date($article);
$item['title'] = $this->extract_article_rating($article) . ' : ' . $this->extract_article_summary($article);
$item['uri'] = $this->extract_article_uri($article);
if($full)
$item['content'] = $this->extract_full_description($item['uri']);
else
$item['content'] = $this->extract_article_description($article);
$this->items[] = $item;
}
}
public function getCacheDuration(){
return 86400; // 1 day
}
/**
* Fixes relative URLs in the given text
*/
private function fix_url($text){
return preg_replace('/href=(\'|\")\//i', 'href="'.$this->uri, $text);
}
/**
* Encodes unmlauts in the given text
*/
private function encode_umlauts($text){
$umlauts = Array("/ä/","/ö/","/ü/","/Ä/","/Ö/","/Ü/","/ß/");
$replace = Array("ae","oe","ue","Ae","Oe","Ue","ss");
return preg_replace($umlauts, $replace, $text);
}
/**
* Returns the company name from the review html
*/
private function extract_company_name($html){
$panel = $html->find('div.panel', 0);
if($panel === false)
$this->returnServerError('Cannot find panel for company name!');
$company_name = $panel->find('h1', 0);
if($company_name === false)
$this->returnServerError('Cannot find company name!');
return $company_name->plaintext;
}
/**
* Returns the date from a given article
*/
private function extract_article_date($article){
// They conviniently provide a time attribute for us :)
$date = $article->find('time[itemprop=dtreviewed]', 0);
if($date === false)
$this->returnServerError('Cannot find article date!');
return strtotime($date->datetime);
}
/**
* Returns the rating from a given article
*/
private function extract_article_rating($article){
$rating = $article->find('span.rating', 0);
if($rating === false)
$this->returnServerError('Cannot find article rating!');
return $rating->getAttribute('aria-label');
}
/**
* Returns the summary from a given article
*/
private function extract_article_summary($article){
$summary = $article->find('[itemprop=summary]', 0);
if($summary === false)
$this->returnServerError('Cannot find article summary!');
return strip_tags($summary->innertext);
}
/**
* Returns the URI from a given article
*/
private function extract_article_uri($article){
// Notice: This first part is the same as in extract_article_summary!
$summary = $article->find('[itemprop=summary]', 0);
if($summary === false)
$this->returnServerError('Cannot find article summary!');
$anchor = $summary->find('a', 0);
if($anchor === false)
$this->returnServerError('Cannot find article URI!');
return $this->uri . $anchor->href;
}
/**
* Returns the position of the author from a given article
*/
private function extract_article_author_position($article){
// We need to parse the aside manually
$aside = $article->find('aside', 0);
if($aside === false)
$this->returnServerError('Cannot find article author information!');
// Go through all h2 elements to find index of required span (I know... it's stupid)
$author_position = 'Unknown';
foreach($aside->find('h2') as $subject){
if(stristr(strtolower($subject->plaintext), 'position')){ /* This works for at, ch, de, us */
$author_position = $subject->next_sibling()->plaintext;
break;
}
}
return $author_position;
}
/**
* Returns the description from a given article
*/
private function extract_article_description($article){
$description = $article->find('div[itemprop=description]', 0);
if($description === false)
$this->returnServerError('Cannot find article description!');
return $this->fix_url($description->innertext);
}
/**
* Returns the full description from a given uri
*/
private function extract_full_description($uri){
// Load full article
if($this->get_cached_time($uri) <= strtotime('-24 hours'))
$this->remove_from_cache($uri);
$html = $this->get_cached($uri);
if($html === false)
$this->returnServerError('Could not load full description!');
// Find the article
$article = $html->find('article', 0);
if($article === false)
$this->returnServerError('Cannot find article!');
// Luckily they use the same layout for the review overview and full article pages :)
return $this->extract_article_description($article);
}
}

81
bridges/LWNprevBridge.php
View File

@ -1,19 +1,12 @@
<?php
/**
* RssBridgeLWNprev
*
* @name LWNPrev Bridge
* @description Returns the articles from the previous LWN.net edition
*/
class LWNprevBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = 'Pierre Mazière';
$this->name = 'LWN Free Weekly Edition';
$this->uri = 'https://lwn.net/free/bigpage';
$this->description = 'LWN Free Weekly Edition available one week late';
$this->update = '2016-19-01';
public $maintainer = 'Pierre Mazière';
public $name = 'LWN Free Weekly Edition';
public $uri = 'https://lwn.net/';
public $description = 'LWN Free Weekly Edition available one week late';
function getURI(){
return $this->uri.'free/bigpage';
}
private function jumpToNextTag(&$node){
@ -36,39 +29,26 @@ class LWNprevBridge extends BridgeAbstract{
}
}
public function collectData(array $param){
public function collectData(){
// Because the LWN page is written in loose HTML and not XHTML,
// Simple HTML Dom is not accurate enough for the job
$uri='https://lwn.net/free/bigpage';
$context=null;
if(defined('PROXY_URL')) {
$context = array(
'http' => array(
'proxy' => PROXY_URL,
'request_fulluri' => true,
),
);
$context = stream_context_create($context);
}
$html=file_get_contents($uri, false, $context)
or $this->returnError('No results for LWNprev', 404);
$content=$this->getContents($this->getURI())
or $this->returnServerError('No results for LWNprev');
libxml_use_internal_errors(true);
$html=DOMDocument::loadHTML($html);
$html=new DOMDocument();
$html->loadHTML($content);
libxml_clear_errors();
$cat1='';
$cat2='';
$realURI='https://lwn.net';
foreach($html->getElementsByTagName('a') as $a){
if($a->textContent==='Multi-page format'){
break;
}
}
$realURI.=$a->getAttribute('href');
$realURI=$this->uri.$a->getAttribute('href');
$URICounter=0;
$edition=$html->getElementsByTagName('h1')->item(0)->textContent;
@ -81,34 +61,34 @@ class LWNprevBridge extends BridgeAbstract{
continue;
}
$item = new \Item();
$item = array();
$h2NextSibling=$h2->nextSibling;
$this->jumpToNextTag($h2NextSibling);
switch($h2NextSibling->getAttribute('class')){
case 'FeatureByline':
$item->name=$h2NextSibling->getElementsByTagName('b')->item(0)->textContent;
$item['author']=$h2NextSibling->getElementsByTagName('b')->item(0)->textContent;
break;
case 'GAByline':
$text=$h2NextSibling->textContent;
$item->name=substr($text,strpos($text,'by '));
$item['author']=substr($text,strpos($text,'by '));
break;
default:
$item->name='LWN';
$item['author']='LWN';
break;
};
$h2FirstChild=$h2->firstChild;
$this->jumpToNextTag($h2FirstChild);
if($h2FirstChild->tagName==='a'){
$item->uri='https://lwn.net'.$h2FirstChild->getAttribute('href');
if($h2FirstChild->nodeName==='a'){
$item['uri']=$this->uri.$h2FirstChild->getAttribute('href');
}else{
$item->uri=$realURI.'#'.$URICounter;
$item['uri']=$realURI.'#'.$URICounter;
}
$URICounter++;
$item->timestamp=$editionTimeStamp+$URICounter;
$item['timestamp']=$editionTimeStamp+$URICounter;
$h2PrevSibling=$h2->previousSibling;
$this->jumpToPreviousTag($h2PrevSibling);
@ -131,11 +111,11 @@ class LWNprevBridge extends BridgeAbstract{
}
$h2PrevSibling=null;
$item->title='';
$item['title']='';
if(!empty($cat1)){
$item->title.='['.$cat1.($cat2?'/'.$cat2:'').'] ';
$item['title'].='['.$cat1.($cat2?'/'.$cat2:'').'] ';
}
$item->title.=$h2->textContent;
$item['title'].=$h2->textContent;
$node=$h2;
$content='';
@ -145,8 +125,9 @@ class LWNprevBridge extends BridgeAbstract{
if(
!$node || (
$node->nodeType!==XML_TEXT_NODE && (
$node->tagName==='h2' ||
in_array($node->getAttribute('class'),array('Cat1HL','Cat2HL'))
$node->nodeName==='h2' ||
(!is_null($node->attributes) && !is_null($class=$node->attributes->getNamedItem('class')) &&
in_array($class->nodeValue,array('Cat1HL','Cat2HL')))
)
)
){
@ -155,19 +136,11 @@ class LWNprevBridge extends BridgeAbstract{
$content.=$node->C14N();
}
}
$item->content=$content;
$item['content']=$content;
$this->items[]=$item;
}
}
public function getName(){
return 'LWN Free Weekly Edition';
}
public function getURI(){
return 'https://lwn.net/free/bigpage';
}
public function getCacheDuration(){
return 604800; // one week
}

399
bridges/LeBonCoinBridge.php
View File

@ -1,245 +1,152 @@
<?php
class LeBonCoinBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "16mhz";
public $name = "LeBonCoin";
public $uri = "http://www.leboncoin.fr/";
public $description = "Returns most recent results from LeBonCoin for a region, and optionally a category and a keyword .";
$this->maintainer = "16mhz";
$this->name = "LeBonCoin";
$this->uri = "http://www.leboncoin.fr";
$this->description = "Returns most recent results from LeBonCoin for a region, and optionally a category and a keyword .";
$this->update = "2016-05-02";
public $parameters = array( array(
'k'=>array('name'=>'Mot Clé'),
'r'=>array(
'name'=>'Région',
'type'=>'list',
'values'=>array(
'Alsace'=>'alsace',
'Aquitaine'=>'aquitaine',
'Auvergne'=>'auvergne',
'Basse Normandie'=>'basse_normandie',
'Bourgogne'=>'bourgogne',
'Bretagne'=>'bretagne',
'Centre'=>'centre',
'Champagne Ardenne'=>'champagne_ardenne',
'Corse'=>'corse',
'Franche Comté'=>'franche_comte',
'Haute Normandie'=>'haute_normandie',
'Ile de France'=>'ile_de_france',
'Languedoc Roussillon'=>'languedoc_roussillon',
'Limousin'=>'limousin',
'Lorraine'=>'lorraine',
'Midi Pyrénées'=>'midi_pyrenees',
'Nord Pas De Calais'=>'nord_pas_de_calais',
'Pays de la Loire'=>'pays_de_la_loire',
'Picardie'=>'picardie',
'Poitou Charentes'=>'poitou_charentes',
'Provence Alpes Côte d\'Azur'=>'provence_alpes_cote_d_azur',
'Rhône-Alpes'=>'rhone_alpes',
'Guadeloupe'=>'guadeloupe',
'Martinique'=>'martinique',
'Guyane'=>'guyane',
'Réunion'=>'reunion'
)
),
'c'=>array(
'name'=>'Catégorie',
'type'=>'list',
'values'=>array(
'TOUS'=>'',
'EMPLOI'=>'_emploi_',
'VEHICULES'=>array(
'Tous'=>'_vehicules_',
'Voitures'=>'voitures',
'Motos'=>'motos',
'Caravaning'=>'caravaning',
'Utilitaires'=>'utilitaires',
'Équipement Auto'=>'equipement_auto',
'Équipement Moto'=>'equipement_moto',
'Équipement Caravaning'=>'equipement_caravaning',
'Nautisme'=>'nautisme',
'Équipement Nautisme'=>'equipement_nautisme'
),
'IMMOBILIER'=>array(
'Tous'=>'_immobilier_',
'Ventes immobilières'=>'ventes_immobilieres',
'Locations'=>'locations',
'Colocations'=>'colocations',
'Bureaux & Commerces'=>'bureaux_commerces'
),
'VACANCES'=>array(
'Tous'=>'_vacances_',
'Location gîtes'=>'locations_gites',
'Chambres d\'hôtes'=>'chambres_d_hotes',
'Campings'=>'campings',
'Hôtels'=>'hotels',
'Hébergements insolites'=>'hebergements_insolites'
),
'MULTIMEDIA'=>array(
'Tous'=>'_multimedia_',
'Informatique'=>'informatique',
'Consoles & Jeux vidéo'=>'consoles_jeux_video',
'Image & Son'=>'image_son',
'Téléphonie'=>'telephonie'
),
'LOISIRS'=>array(
'Tous'=>'_loisirs_',
'DVD / Films'=>'dvd_films',
'CD / Musique'=>'cd_musique',
'Livres'=>'livres',
'Animaux'=>'animaux',
'Vélos'=>'velos',
'Sports & Hobbies'=>'sports_hobbies',
'Instruments de musique'=>'instruments_de_musique',
'Collection'=>'collection',
'Jeux & Jouets'=>'jeux_jouets',
'Vins & Gastronomie'=>'vins_gastronomie'
),
'MATÉRIEL PROFESSIONNEL'=>array(
'Tous'=>'_materiel_professionnel_',
'Matériel Agricole'=>'mateiel_agricole',
'Transport - Manutention'=>'transport_manutention',
'BTP - Chantier - Gros-œuvre'=>'btp_chantier_gros_oeuvre',
'Outillage - Matériaux 2nd-œuvre'=>'outillage_materiaux_2nd_oeuvre',
'Équipements Industriels'=>'equipement_industriels',
'Restauration - Hôtellerie'=>'restauration_hotellerie',
'Fournitures de Bureau'=>'fournitures_de_bureau',
'Commerces & Marchés'=>'commerces_marches',
'Matériel médical'=>'materiel_medical'
),
'SERVICES'=>array(
'Tous'=>'_services_',
'Prestations de services'=>'prestations_de_services',
'Billetterie'=>'billetterie',
'Évènements'=>'evenements',
'Cours particuliers'=>'cours_particuliers',
'Covoiturage'=>'covoiturage'
),
'MAISON'=>array(
'Tous'=>'_maison_',
'Ameublement'=>'ameublement',
'Électroménager'=>'electromenager',
'Arts de la table'=>'arts_de_la_table',
'Décoration'=>'decoration',
'Linge de maison'=>'linge_de_maison',
'Bricolage'=>'bricolage',
'Jardinage'=>'jardinage',
'Vêtements'=>'vetements',
'Chaussures'=>'chaussures',
'Accessoires & Bagagerie'=>'accessoires_bagagerie',
'Montres & Bijoux'=>'montres_bijoux',
'Équipement bébé'=>'equipement_bebe',
'Vêtements bébé'=>'vetements_bebe'
),
'AUTRES'=>'autres'
)
)
)
);
$this->parameters[] =
'[
public function collectData(){
{
"name" : "Mot Clé",
"identifier" : "k"
},
$category=$this->getInput('c');
if (empty($category)){
$category='annonces';
}
{
"name" : "Région",
"identifier" : "r",
"type" : "list",
"values" : [
{
"name" : "Alsace",
"value" : "alsace"
},
{
"name" : "Aquitaine",
"value" : "aquitaine"
},
{
"name" : "Auvergne",
"value" : "auvergne"
},
{
"name" : "Basse Normandie",
"value" : "basse_normandie"
},
{
"name" : "Bourgogne",
"value" : "bourgogne"
},
{
"name" : "Bretagne",
"value" : "bretagne"
},
{
"name" : "Centre",
"value" : "centre"
},
{
"name" : "Champagne Ardenne",
"value" : "champagne_ardenne"
},
{
"name" : "Corse",
"value" : "corse"
},
{
"name" : "Franche Comté",
"value" : "franche_comte"
},
{
"name" : "Haute Normandie",
"value" : "haute_normandie"
},
{
"name" : "Ile de France",
"value" : "ile_de_france"
},
{
"name" : "Languedoc Roussillon",
"value" : "languedoc_roussillon"
},
{
"name" : "Limousin",
"value" : "limousin"
},
{
"name" : "Lorraine",
"value" : "lorraine"
},
{
"name" : "Midi Pyrénées",
"value" : "midi_pyrenees"
},
{
"name" : "Nord Pas De Calais",
"value" : "nord_pas_de_calais"
},
{
"name" : "Pays de la Loire",
"value" : "pays_de_la_loire"
},
{
"name" : "Picardie",
"value" : "picardie"
},
{
"name" : "Poitou Charentes",
"value" : "poitou_charentes"
},
{
"name" : "Provence Alpes Côte d\'Azur",
"value" : "provence_alpes_cote_d_azur"
},
{
"name" : "Rhône-Alpes",
"value" : "rhone_alpes"
},
{
"name" : "Guadeloupe",
"value" : "guadeloupe"
},
{
"name" : "Martinique",
"value" : "martinique"
},
{
"name" : "Guyane",
"value" : "guyane"
},
{
"name" : "Réunion",
"value" : "reunion"
}
]
},
{
"name" : "Catégorie",
"identifier" : "c",
"type" : "list",
"values" : [
{ "name" : "---- Select ----", "value" : "" },
{ "name" : "", "value" : "" },
{ "name" : "EMPLOI", "value" : "_emploi_" },
{ "name" : "", "value" : "" },
{ "name" : "VEHICULES", "value" : "_vehicules_" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Voitures", "value" : "voitures" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Motos", "value" : "motos" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Caravaning", "value" : "caravaning" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Utilitaires", "value" : "utilitaires" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Equipement Auto", "value" : "equipement_auto" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Equipement Moto", "value" : "equipement_moto" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Equipement Caravaning", "value" : "equipement_caravaning" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Nautisme", "value" : "nautisme" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Equipement Nautisme", "value" : "equipement_nautisme" },
{ "name" : "", "value" : "" },
{ "name" : "IMMOBILIER", "value" : "_immobilier_" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Ventes immobilieres", "value" : "ventes_immobilieres" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Locations", "value" : "locations" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Colocations", "value" : "colocations" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Bureaux &amp; Commerces", "value" : "bureaux_commerces" },
{ "name" : "", "value" : "" },
{ "name" : "VACANCES", "value" : "_vacances_" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Locations gites", "value" : "locations_gites" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Chambres d\'hôtes", "value" : "chambres_d_hotes" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Campings", "value" : "campings" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Hôtels", "value" : "hotels" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Hébergements insolites", "value" : "hebergements_insolites" },
{ "name" : "", "value" : "" },
{ "name" : "MULTIMEDIA", "value" : "_multimedia_" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Informatique", "value" : "informatique" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Consoles & Jeux vidéo", "value" : "consoles_jeux_video" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Image & Son", "value" : "image_son" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Téléphonie", "value" : "telephonie" },
{ "name" : "", "value" : "" },
{ "name" : "LOISIRS", "value" : "_loisirs_" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;DVD / Films", "value" : "dvd_films" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;CD / Musique", "value" : "cd_musique" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Livres", "value" : "livres" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Animaux", "value" : "animaux" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Vélos", "value" : "velos" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Sports & Hobbies", "value" : "sports_hobbies" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Instruments de musique", "value" : "instruments_de_musique" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Collection", "value" : "collection" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Jeux & Jouets", "value" : "jeux_jouets" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Vins & Gastronomie", "value" : "vins_gastronomie" },
{ "name" : "", "value" : "" },
{ "name" : "MATERIEL PROFESSIONNEL", "value" : "_materiel_professionnel_" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Materiel Agricole", "value" : "materiel_agricole" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Transport - Manutention", "value" : "transport_manutention" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;BTP - Chantier - Gros-oeuvre", "value" : "btp_chantier_gros_oeuvre" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Outillage - Materiaux 2nd-oeuvre", "value" : "outillage_materiaux_2nd_oeuvre" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Equipements Industriels", "value" : "equipements_industriels" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Restauration - Hôtellerie", "value" : "restauration_hotellerie" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Fournitures de Bureau", "value" : "fournitures_de_bureau" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Commerces & Marchés", "value" : "commerces_marches" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Matériel médical", "value" : "materiel_medical" },
{ "name" : "", "value" : "" },
{ "name" : "SERVICES", "value" : "_services_" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Prestations de services", "value" : "prestations_de_services" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Billetterie", "value" : "billetterie" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Evénements", "value" : "evenements" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Cours particuliers", "value" : "cours_particuliers" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Covoiturage", "value" : "covoiturage" },
{ "name" : "", "value" : "" },
{ "name" : "MAISON", "value" : "_maison_" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Ameublement", "value" : "ameublement" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Electroménager", "value" : "electromenager" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Arts de la table", "value" : "arts_de_la_table" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Décoration", "value" : "decoration" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Linge de maison", "value" : "linge_de_maison" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Bricolage", "value" : "bricolage" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Jardinage", "value" : "jardinage" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Vêtements", "value" : "vetements" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Chaussures", "value" : "chaussures" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Accessoires & Bagagerie", "value" : "accessoires_bagagerie" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Montres & Bijoux", "value" : "montres_bijoux" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Equipement bébé", "value" : "equipement_bebe" },
{ "name" : "&nbsp;&nbsp;&nbsp;&nbsp;Vêtements bébé", "value" : "vetements_bebe" },
{ "name" : "", "value" : "" },
{ "name" : "AUTRES", "value" : "autres" }
]
}
]';
}
public function collectData(array $param){
$html = '';
if (empty($param['c'])) {
$link = 'http://www.leboncoin.fr/annonces/offres/' . $param['r'] . '/?f=a&th=1&q=' . urlencode($param['k']);
}
else {
$link = 'http://www.leboncoin.fr/' . $param['c'] . '/offres/' . $param['r'] . '/?f=a&th=1&q=' . urlencode($param['k']);
}
$html = $this->file_get_html($link) or $this->returnError('Could not request LeBonCoin.', 404);
$html = $this->getSimpleHTMLDOM(
$this->uri.$category.'/offres/' . $this->getInput('r') . '/?'
.'f=a&th=1&'
.'q=' . urlencode($this->getInput('k'))
) or $this->returnServerError('Could not request LeBonCoin.');
$list = $html->find('.tabsContent', 0);
if($list === NULL) {
@ -252,8 +159,8 @@ class LeBonCoinBridge extends BridgeAbstract{
$element = $element->find('a', 0);
$item = new \Item();
$item->uri = $element->href;
$item = array();
$item['uri'] = $element->href;
$title = html_entity_decode($element->getAttribute('title'));
$content_image = $element->find('div.item_image', 0)->find('.lazyload', 0);
@ -270,21 +177,9 @@ class LeBonCoinBridge extends BridgeAbstract{
$price = $detailsList->find('h3.item_price', 0);
$content .= $price === NULL ? '' : $price->plaintext;
$item->title = $title;
$item->content = $content . $date;
$item['title'] = $title;
$item['content'] = $content . $date;
$this->items[] = $item;
}
}
public function getName(){
return 'LeBonCoin';
}
public function getURI(){
return 'http://www.leboncoin.fr';
}
public function getCacheDuration(){
return 3600; // 1 hour
}
}
}

101
bridges/LeJournalDuGeekBridge.php
View File

@ -1,69 +1,58 @@
<?php
class LeJournalDuGeekBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "polopollo";
$this->name = "journaldugeek.com (FR)";
$this->uri = "http://www.journaldugeek.com/";
$this->description = "Returns the 5 newest posts from LeJournalDuGeek (full text).";
$this->update = "2014-07-14";
public $maintainer = "polopollo";
public $name = "journaldugeek.com (FR)";
public $uri = "http://www.journaldugeek.com/";
public $description = "Returns the 5 newest posts from LeJournalDuGeek (full text).";
private function LeJournalDuGeekStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
public function collectData(array $param){
private function LeJournalDuGeekExtractContent($url) {
$articleHTMLContent = $this->getSimpleHTMLDOM($url);
$text = $articleHTMLContent->find('div.post-content', 0)->innertext;
function LeJournalDuGeekStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
foreach($articleHTMLContent->find('a.more') as $element) {
if ($element->innertext == "Source") {
$text = $text . '<p><a href="' . $element->href . '">Source : ' . $element->href . '</a></p>';
break;
}
}
function LeJournalDuGeekExtractContent($url) {
$articleHTMLContent = $this->file_get_html($url);
$text = $text.$articleHTMLContent->find('div.post-content', 0)->innertext;
foreach($articleHTMLContent->find('a.more') as $element) {
if ($element->innertext == "Source") {
$text = $text.'<p><a href="'.$element->href.'">Source : '.$element->href.'</a></p>';
break;
}
}
foreach($articleHTMLContent->find('iframe') as $element) {
if (preg_match("/youtube/i", $element->src)) {
$text = $text.'// An IFRAME to Youtube was included in the article: <a href="'.$element->src.'">'.$element->src.'</a><br>';
}
}
foreach($articleHTMLContent->find('iframe') as $element) {
if (preg_match("/youtube/i", $element->src)) {
$text = $text . '// An IFRAME to Youtube was included in the article: <a href="' . $element->src . '">' . $element->src . '</a><br>';
}
}
$text = strip_tags($text, '<p><b><a><blockquote><img><em><br/><br><ul><li>');
return $text;
}
$text = preg_replace('#<script(.*?)>(.*?)</script>#is', '', $text);
$text = strip_tags($text, '<p><b><a><blockquote><img><em><br/><br><ul><li>');
return $text;
}
$rssFeed = $this->file_get_html('http://www.journaldugeek.com/rss') or $this->returnError('Could not request http://www.journaldugeek.com/rss', 404);
$limit = 0;
public function collectData(){
$rssFeed = $this->getSimpleHTMLDOM($this->uri.'rss')
or $this->returnServerError('Could not request '.$this->uri.'/rss');
$limit = 0;
foreach($rssFeed->find('item') as $element) {
if($limit < 5) {
$item = new \Item();
$item->title = LeJournalDuGeekStripCDATA($element->find('title', 0)->innertext);
$item->uri = LeJournalDuGeekStripCDATA($element->find('guid', 0)->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = LeJournalDuGeekExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
}
foreach($rssFeed->find('item') as $element) {
if($limit < 5) {
$item = array();
$item['title'] = $this->LeJournalDuGeekStripCDATA($element->find('title', 0)->innertext);
$item['uri'] = $this->LeJournalDuGeekStripCDATA($element->find('guid', 0)->plaintext);
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $this->LeJournalDuGeekExtractContent($item['uri']);
$this->items[] = $item;
$limit++;
}
}
}
}
public function getName(){
return 'LeJournalDuGeek';
}
public function getURI(){
return 'http://www.journaldugeek.com/';
}
public function getCacheDuration(){
return 1800; // 30min
}
public function getCacheDuration(){
return 1800; // 30min
}
}

44
bridges/LeMondeInformatiqueBridge.php
View File

@ -1,17 +1,12 @@
<?php
class LeMondeInformatiqueBridge extends BridgeAbstract {
public function loadMetadatas() {
public $maintainer = "ORelio";
public $name = "Le Monde Informatique";
public $uri = "http://www.lemondeinformatique.fr/";
public $description = "Returns the newest articles.";
$this->maintainer = "ORelio";
$this->name = "Le Monde Informatique";
$this->uri = "http://www.lemondeinformatique.fr/";
$this->description = "Returns the newest articles.";
$this->update = "2016-01-28";
}
public function collectData(array $param) {
public function collectData(){
function StripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
@ -33,8 +28,9 @@ class LeMondeInformatiqueBridge extends BridgeAbstract {
return $article_html;
}
$feedUrl = 'http://www.lemondeinformatique.fr/rss/rss.xml';
$html = $this->file_get_html($feedUrl) or $this->returnError('Could not request LeMondeInformatique: '.$feedUrl, 500);
$html = $this->getSimpleHTMLDOM($this->uri.'rss/rss.xml')
or $this->returnServerError('Could not request LeMondeInformatique: '
.$this->uri.'rss/rss.xml');
$limit = 0;
foreach($html->find('item') as $element) {
@ -44,33 +40,23 @@ class LeMondeInformatiqueBridge extends BridgeAbstract {
$article_uri = $element->innertext;
$article_uri = substr($article_uri, strpos($article_uri, '<link>') + 6);
$article_uri = substr($article_uri, 0, strpos($article_uri, '</link>'));
$article_html = $this->file_get_html($article_uri) or $this->returnError('Could not request LeMondeInformatique: '.$article_uri, 500);
$thumbnailUri = $article_html->find('div#article', 0)->find('img#illustration', 0)->src;
$article_html = $this->getSimpleHTMLDOM($article_uri) or $this->returnServerError('Could not request LeMondeInformatique: '.$article_uri);
$article_content = CleanArticle($article_html->find('div#article', 0)->innertext);
$article_title = $article_html->find('h1.cleanprint-title', 0)->plaintext;
//Build and add final item
$item = new \Item();
$item->uri = $article_uri;
$item->thumbnailUri = $thumbnailUri;
$item->title = $article_title;
$item->author = StripCDATA($element->find('dc:creator', 0)->innertext);
$item->timestamp = strtotime($element->find('dc:date', 0)->plaintext);
$item->content = $article_content;
$item = array();
$item['uri'] = $article_uri;
$item['title'] = $article_title;
$item['author'] = StripCDATA($element->find('dc:creator', 0)->innertext);
$item['timestamp'] = strtotime($element->find('dc:date', 0)->plaintext);
$item['content'] = $article_content;
$this->items[] = $item;
$limit++;
}
}
}
public function getName() {
return 'Le Monde Informatique';
}
public function getURI() {
return 'http://www.lemondeinformatique.fr/';
}
public function getCacheDuration() {
return 1800; // 30 minutes
}

55
bridges/LeMotDuJourBridge.php
View File

@ -1,55 +0,0 @@
<?php
class LeMotDuJourBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "qwertygc";
$this->name = "LeMotDuJour Bridge";
$this->uri = "http://www.lemotdujour.com/";
$this->description = "Returns the newest articles.";
$this->update = "2014-05-25";
}
public function collectData(array $param){
function StripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function ExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = $html2->find('div.single-contenu', 0)->innertext;
return $text;
}
$html = $this->file_get_html('http://feeds2.feedburner.com/lemotdujour/lemotdujour') or $this->returnError('Could not request LeMotDuJour.', 404);
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 10) {
$item = new \Item();
$item->title = StripCDATA($element->find('title', 0)->innertext);
$item->uri = StripCDATA($element->find('guid', 0)->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = ExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'LeMotDuJour Bridge';
}
public function getURI(){
return 'http://lemotdujour.com/';
}
public function getCacheDuration(){
return 3600*2; // 2 hours
// return 0; // 2 hours
}
}

44
bridges/Les400Culs.php
View File

@ -1,44 +0,0 @@
<?php
define("SEXE", "http://sexes.blogs.liberation.fr");
define("SEXE_FEED", "http://sexes.blogs.liberation.fr/feeds/");
class Les400Culs extends RssExpander{
public function loadMetadatas() {
$this->maintainer = "unknown";
$this->name = "Les 400 Culs";
$this->uri = "http://sexes.blogs.liberation.fr";
$this->description = "La planete sexe vue par Agnes Girard via rss-bridge";
$this->update = "20/02/2014";
}
public function collectData(array $param){
parent::collectExpandableDatas($param, SEXE_FEED);
}
protected function parseRSSItem($newsItem) {
$item = new Item();
$item->title = trim((string) $newsItem->title);
// $this->message("browsing item ".var_export($newsItem, true));
if(empty($newsItem->guid)) {
$item->uri = (string) $newsItem->link;
} else {
$item->uri = (string) $newsItem->guid;
}
// now load that uri from cache
// $this->message("now loading page ".$item->uri);
// $articlePage = str_get_html($this->get_cached($item->uri));
// $content = $articlePage->find('.post-container', 0);
$item->content = (string) $newsItem->description;
$item->name = (string) $newsItem->author;
$item->timestamp = $this->RSS_2_0_time_to_timestamp($newsItem);
return $item;
}
public function getCacheDuration(){
return 7200; // 2h hours
}
}

36
bridges/Les400CulsBridge.php Normal file
View File

@ -0,0 +1,36 @@
<?php
class Les400CulsBridge extends RssExpander{
public $maintainer = "unknown";
public $name = "Les 400 Culs";
public $uri = "http://sexes.blogs.liberation.fr/";
public $description = "La planete sexe vue par Agnes Girard via rss-bridge";
public function collectData(){
$this->collectExpandableDatas($this->uri.'feeds/');
}
protected function parseRSSItem($newsItem) {
$item = array();
$item['title'] = trim((string) $newsItem->title);
$this->debugMessage("browsing item ".var_export($newsItem, true));
if(empty($newsItem->guid)) {
$item['uri'] = (string) $newsItem->link;
} else {
$item['uri'] = (string) $newsItem->guid;
}
// now load that uri from cache
$this->debugMessage("now loading page ".$item['uri']);
// $articlePage = $this->get_cached($item['uri']);
// $content = $articlePage->find('.post-container', 0);
$item['content'] = (string) $newsItem->description;
$item['author'] = (string) $newsItem->author;
$item['timestamp'] = $this->RSS_2_0_time_to_timestamp($newsItem);
return $item;
}
public function getCacheDuration(){
return 7200; // 2h hours
}
}

52
bridges/LesJoiesDuCodeBridge.php
View File

@ -1,25 +1,21 @@
<?php
class LesJoiesDuCodeBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "superbaillot.net";
public $name = "Les Joies Du Code";
public $uri = "http://lesjoiesducode.fr/";
public $description = "LesJoiesDuCode";
$this->maintainer = "superbaillot.net";
$this->name = "Les Joies Du Code";
$this->uri = "http://lesjoiesducode.fr/";
$this->description = "LesJoiesDuCode";
$this->update = "04/02/2015";
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Could not request LesJoiesDuCode.');
}
public function collectData(array $param){
$html = $this->file_get_html('http://lesjoiesducode.fr/') or $this->returnError('Could not request LesJoiesDuCode.', 404);
foreach($html->find('div.blog-post') as $element) {
$item = new Item();
$item = array();
$temp = $element->find('h1 a', 0);
$titre = html_entity_decode($temp->innertext);
$url = $temp->href;
$temp = $element->find('div.blog-post-content', 0);
// retrieve .gif instead of static .jpg
@ -29,38 +25,26 @@ class LesJoiesDuCodeBridge extends BridgeAbstract{
$image->src = $img_src;
}
$content = $temp->innertext;
$auteur = $temp->find('i', 0);
$pos = strpos($auteur->innertext, "by");
if($pos > 0)
{
$auteur = trim(str_replace("*/", "", substr($auteur->innertext, ($pos + 2))));
$item->name = $auteur;
$item['author'] = $auteur;
}
$item->content .= trim($content);
$item->uri = $url;
$item->title = trim($titre);
$item['content'] .= trim($content);
$item['uri'] = $url;
$item['title'] = trim($titre);
$this->items[] = $item;
}
}
public function getName(){
return 'Les Joies Du Code';
}
public function getURI(){
return 'http://lesjoiesducode.fr/';
}
public function getCacheDuration(){
return 7200; // 2h hours
}
public function getDescription(){
return "Les Joies Du Code via rss-bridge";
}
}
?>

46
bridges/LichessBridge.php
View File

@ -1,33 +1,30 @@
<?php
class LichessBridge extends BridgeAbstract
class LichessBridge extends HttpCachingBridgeAbstract
{
public function loadMetadatas()
{
$this->maintainer = 'AmauryCarrade';
$this->name = $this->getName() . ' Bridge';
$this->uri = $this->getURI();
$this->description = 'Returns the 5 newest posts from the Lichess blog (full text)';
$this->update = "2016-03-22";
}
public $maintainer = 'AmauryCarrade';
public $name = 'Lichess Blog';
public $uri = 'http://fr.lichess.org/blog';
public $description = 'Returns the 5 newest posts from the Lichess blog (full text)';
public function collectData(array $param)
public function collectData()
{
$xml_feed = $this->file_get_html('http://fr.lichess.org/blog.atom') or $this->returnError('Could not retrieve Lichess blog feed.', 404);
$xml_feed = $this->getSimpleHTMLDOM($this->uri.'.atom')
or $this->returnServerError('Could not retrieve Lichess blog feed.');
$posts_loaded = 0;
foreach($xml_feed->find('entry') as $entry)
{
if ($posts_loaded < 5)
{
$item = new \Item();
$item = array();
$item->title = html_entity_decode($entry->find('title', 0)->innertext);
$item->author = $entry->find('author', 0)->find('name', 0)->innertext;
$item->uri = $entry->find('id', 0)->plaintext;
$item->timestamp = strtotime($entry->find('published', 0)->plaintext);
$item['title'] = html_entity_decode($entry->find('title', 0)->innertext);
$item['author'] = $entry->find('author', 0)->find('name', 0)->innertext;
$item['uri'] = $entry->find('id', 0)->plaintext;
$item['timestamp'] = strtotime($entry->find('published', 0)->plaintext);
$item->content = $this->retrieve_lichess_post($item->uri);
$item['content'] = $this->retrieve_lichess_post($item['uri']);
$this->items[] = $item;
$posts_loaded++;
@ -37,7 +34,10 @@ class LichessBridge extends BridgeAbstract
private function retrieve_lichess_post($blog_post_uri)
{
$blog_post_html = $this->file_get_html($blog_post_uri);
if($this->get_cached_time($blog_post_uri) <= strtotime('-24 hours'))
$this->remove_from_cache($blog_post_uriuri);
$blog_post_html = $this->get_cached($blog_post_uri);
$blog_post_div = $blog_post_html->find('#lichess_blog', 0);
$post_chapo = $blog_post_div->find('.shortlede', 0)->innertext;
@ -48,14 +48,4 @@ class LichessBridge extends BridgeAbstract
return $content;
}
public function getName()
{
return 'Lichess Blog';
}
public function getURI()
{
return 'http://lichess.org/blog';
}
}

52
bridges/LinkedInCompany.php
View File

@ -1,52 +0,0 @@
<?php
class LinkedInCompany extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "regisenguehard";
$this->name = "LinkedIn Company";
$this->uri = "https://www.linkedin.com/";
$this->description = "Returns most recent actus from Company on LinkedIn. (https://www.linkedin.com/company/<strong style=\"font-weight:bold;\">apple</strong>)";
$this->update = "2015-12-22";
$this->parameters[] =
'[
{
"name" : "Company name",
"identifier" : "c"
}
]';
}
public function collectData(array $param){
$html = '';
$link = 'https://www.linkedin.com/company/'.$param[c];
$html = $this->file_get_html($link) or $this->returnError('Could not request LinkedIn.', 404);
foreach($html->find('//*[@id="my-feed-post"]/li') as $element) {
$title = $element->find('span.share-body', 0)->innertext;
if ($title) {
$item = new \Item();
$item->uri = $link;
$item->title = mb_substr(strip_tags($element->find('span.share-body', 0)->innertext), 0 ,100);
$item->content = strip_tags($element->find('span.share-body', 0)->innertext);
$item->thumbnailUri = htmlspecialchars_decode($element->find('img', 0)->attr['data-li-lazy-load-src']);
$this->items[] = $item;
$i++;
}
}
}
public function getName(){
return 'LinkedIn';
}
public function getURI(){
return 'https://www.linkedin.com';
}
public function getCacheDuration(){
return 21600; // 6 hours
}
}

39
bridges/LinkedInCompanyBridge.php Normal file
View File

@ -0,0 +1,39 @@
<?php
class LinkedInCompanyBridge extends BridgeAbstract{
public $maintainer = "regisenguehard";
public $name = "LinkedIn Company";
public $uri = "https://www.linkedin.com/";
public $description = "Returns most recent actus from Company on LinkedIn. (https://www.linkedin.com/company/<strong style=\"font-weight:bold;\">apple</strong>)";
public $parameters = array( array(
'c'=>array(
'name'=>'Company name',
'required'=>true
)
));
public function collectData(){
$html = '';
$link = $this->uri.'company/'.$this->getInput('c');
$html = $this->getSimpleHTMLDOM($link)
or $this->returnServerError('Could not request LinkedIn.');
foreach($html->find('//*[@id="my-feed-post"]/li') as $element) {
$title = $element->find('span.share-body', 0)->innertext;
if ($title) {
$item = array();
$item['uri'] = $link;
$item['title'] = mb_substr(strip_tags($element->find('span.share-body', 0)->innertext), 0 ,100);
$item['content'] = strip_tags($element->find('span.share-body', 0)->innertext);
$this->items[] = $item;
$i++;
}
}
}
public function getCacheDuration(){
return 21600; // 6 hours
}
}

72
bridges/LolibooruBridge.php
View File

@ -1,65 +1,45 @@
<?php
class LolibooruBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "mitsukarenai";
public $name = "Lolibooru";
public $uri = "http://lolibooru.moe/";
public $description = "Returns images from given page and tags";
$this->maintainer = "mitsukarenai";
$this->name = "Lolibooru";
$this->uri = "http://lolibooru.moe/";
$this->description = "Returns images from given page and tags";
$this->update = "2015-03-21";
public $parameters = array( array(
'p'=>array(
'name'=>'page',
'defaultValue'=>1,
'type'=>'number'
),
't'=>array('name'=>'tags')
));
$this->parameters[] =
'[
{
"name" : "page",
"identifier" : "p",
"type" : "number"
},
{
"name" : "tags",
"identifier" : "t"
}
]';
public function collectData(){
$html = $this->getSimpleHTMLDOM(
$this->uri.'post?'
.'&page='.$this->getInput('p')
.'&tags='.urlencode($this->getInput('t'))
) or $this->returnServerError('Could not request Lolibooru.');
}
public function collectData(array $param){
$page = 1; $tags = '';
if (isset($param['p'])) {
$page = (int)preg_replace("/[^0-9]/",'', $param['p']);
}
if (isset($param['t'])) {
$tags = urlencode($param['t']);
}
$html = $this->file_get_html("http://lolibooru.moe/post?page=$page&tags=$tags") or $this->returnError('Could not request Lolibooru.', 404);
$input_json = explode('Post.register(', $html);
foreach($input_json as $element)
$data[] = preg_replace('/}\)(.*)/', '}', $element);
unset($data[0]);
foreach($data as $datai) {
$json = json_decode($datai, TRUE);
$item = new \Item();
$item->uri = 'http://lolibooru.moe/post/show/'.$json['id'];
$item->postid = $json['id'];
$item->timestamp = $json['created_at'];
$item->imageUri = $json['file_url'];
$item->thumbnailUri = $json['preview_url'];
$item->title = 'Lolibooru | '.$json['id'];
$item->content = '<a href="' . $item->imageUri . '"><img src="' . $item->thumbnailUri . '" /></a><br>Tags: '.$json['tags'];
$item = array();
$item['uri'] = $this->uri.'post/show/'.$json['id'];
$item['postid'] = $json['id'];
$item['timestamp'] = $json['created_at'];
$item['imageUri'] = $json['file_url'];
$item['title'] = 'Lolibooru | '.$json['id'];
$item['content'] = '<a href="' . $item['imageUri'] . '"><img src="' . $json['preview_url'] . '" /></a><br>Tags: '.$json['tags'];
$this->items[] = $item;
}
}
public function getName(){
return 'Lolibooru';
}
public function getURI(){
return 'http://lolibooru.moe/post';
}
public function getCacheDuration(){
return 1800; // 30 minutes
}

60
bridges/MalikiBridge.php
View File

@ -1,60 +0,0 @@
<?php
class MalikiBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "mitsukarenai";
$this->name = "Maliki";
$this->uri = "http://www.maliki.com/";
$this->description = "Returns Maliki's newest strips";
$this->update = "2014-05-30";
}
public function collectData(array $param){
$html = $this->file_get_html('http://www.maliki.com/') or $this->returnError('Could not request Maliki.', 404);
$count=0;
$latest=1; $latest_title="";
$latest = $html->find('div.conteneur_page a', 1)->href;
$latest_title = $html->find('div.conteneur_page img', 0)->title;
function MalikiExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = 'http://www.maliki.com/'.$html2->find('img', 0)->src;
$text = '<img alt="" src="'.$text.'"/><br>'.$html2->find('div.imageetnews', 0)->plaintext;
return $text;
}
$item = new \Item();
$item->uri = 'http://www.maliki.com/'.$latest;
$item->title = $latest_title;
$item->timestamp = time();
$item->content = MalikiExtractContent($item->uri);
$this->items[] = $item;
foreach($html->find('div.boite_strip') as $element) {
if(!empty($element->find('a',0)->href) and $count < 3) {
$item = new \Item();
$item->uri = 'http://www.maliki.com/'.$element->find('a',0)->href;
$item->title = $element->find('img',0)->title;
$item->timestamp = strtotime(str_replace('/', '-', $element->find('span.stylepetit', 0)->innertext));
$item->content = MalikiExtractContent($item->uri);
$this->items[] = $item;
$count++;
}
}
}
public function getName(){
return 'Maliki';
}
public function getURI(){
return 'http://www.maliki.com/';
}
public function getCacheDuration(){
return 86400*6; // 6 days
}
}

435
bridges/MangareaderBridge.php
View File

@ -2,338 +2,215 @@
define('MANGAREADER_LIMIT', 10); // The default limit
class MangareaderBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "logmanoriginal";
public $name = "Mangareader Bridge";
public $uri = "http://www.mangareader.net/";
public $description = "Returns the latest updates, popular mangas or manga updates (new chapters)";
$this->maintainer = "logmanoriginal";
$this->name = "Mangareader Bridge";
$this->uri = "http://www.mangareader.net";
$this->description = "Returns the latest updates, popular mangas or manga updates (new chapters)";
$this->update = "2016-01-22";
public $parameters = array(
'Get latest updates' => array(),
'Get popular mangas' => array(
'category'=>array(
'name'=>'Category',
'type'=>'list',
'required'=>true,
'values'=>array(
'All'=>'all',
'Action'=>'action',
'Adventure'=>'adventure',
'Comedy'=>'comedy',
'Demons'=>'demons',
'Drama'=>'drama',
'Ecchi'=>'ecchi',
'Fantasy'=>'fantasy',
'Gender Bender'=>'gender-bender',
'Harem'=>'harem',
'Historical'=>'historical',
'Horror'=>'horror',
'Josei'=>'josei',
'Magic'=>'magic',
'Martial Arts'=>'martial-arts',
'Mature'=>'mature',
'Mecha'=>'mecha',
'Military'=>'military',
'Mystery'=>'mystery',
'One Shot'=>'one-shot',
'Psychological'=>'psychological',
'Romance'=>'romance',
'School Life'=>'school-life',
'Sci-Fi'=>'sci-fi',
'Seinen'=>'seinen',
'Shoujo'=>'shoujo',
'Shoujoai'=>'shoujoai',
'Shounen'=>'shounen',
'Shounenai'=>'shounenai',
'Slice of Life'=>'slice-of-life',
'Smut'=>'smut',
'Sports'=>'sports',
'Super Power'=>'super-power',
'Supernatural'=>'supernatural',
'Tragedy'=>'tragedy',
'Vampire'=>'vampire',
'Yaoi'=>'yaoi',
'Yuri'=>'yuri'
),
'exampleValue'=>'All',
'title'=>'Select your category'
)
),
'Get manga updates' => array(
'path'=>array(
'name'=>'Path',
'required'=>true,
'pattern'=>'[a-zA-Z0-9-_]*',
'exampleValue'=>'bleach, umi-no-kishidan',
'title'=>'URL part of desired manga'
),
'limit'=>array(
'name'=>'Limit',
'type'=>'number',
'exampleValue'=>10,
'title'=>'Number of items to return [-1 returns all]'
)
)
);
$this->parameters["Get latest updates"] = '[]';
$this->parameters["Get popular mangas"] =
'[
{
"name" : "Category",
"identifier" : "category",
"type" : "list",
"required" : "true",
"values" : [
{
"name" : "All",
"value" : "all"
},
{
"name" : "Action",
"value" : "action"
},
{
"name" : "Adventure",
"value" : "adventure"
},
{
"name" : "Comedy",
"value" : "comedy"
},
{
"name" : "Demons",
"value" : "demons"
},
{
"name" : "Drama",
"value" : "drama"
},
{
"name" : "Ecchi",
"value" : "ecchi"
},
{
"name" : "Fantasy",
"value" : "fantasy"
},
{
"name" : "Gender Bender",
"value" : "gender-bender"
},
{
"name" : "Harem",
"value" : "harem"
},
{
"name" : "Historical",
"value" : "historical"
},
{
"name" : "Horror",
"value" : "horror"
},
{
"name" : "Josei",
"value" : "josei"
},
{
"name" : "Magic",
"value" : "magic"
},
{
"name" : "Martial Arts",
"value" : "martial-arts"
},
{
"name" : "Mature",
"value" : "mature"
},
{
"name" : "Mecha",
"value" : "mecha"
},
{
"name" : "Military",
"value" : "military"
},
{
"name" : "Mystery",
"value" : "mystery"
},
{
"name" : "One Shot",
"value" : "one-shot"
},
{
"name" : "Psychological",
"value" : "psychological"
},
{
"name" : "Romance",
"value" : "romance"
},
{
"name" : "School Life",
"value" : "school-life"
},
{
"name" : "Sci-Fi",
"value" : "sci-fi"
},
{
"name" : "Seinen",
"value" : "seinen"
},
{
"name" : "Shoujo",
"value" : "shoujo"
},
{
"name" : "Shoujoai",
"value" : "shoujoai"
},
{
"name" : "Shounen",
"value" : "shounen"
},
{
"name" : "Shounenai",
"value" : "shounenai"
},
{
"name" : "Slice of Life",
"value" : "slice-of-life"
},
{
"name" : "Smut",
"value" : "smut"
},
{
"name" : "Sports",
"value" : "sports"
},
{
"name" : "Super Power",
"value" : "super-power"
},
{
"name" : "Supernatural",
"value" : "supernatural"
},
{
"name" : "Tragedy",
"value" : "tragedy"
},
{
"name" : "Vampire",
"value" : "vampire"
},
{
"name" : "Yaoi",
"value" : "yaoi"
},
{
"name" : "Yuri",
"value" : "yuri"
}
],
"exampleValue" : "All",
"title" : "Select your category"
}
]';
$this->parameters["Get manga updates"] =
'[
{
"name" : "Path",
"identifier" : "path",
"type" : "text",
"required" : "true",
"pattern" : "[a-zA-Z0-9-_]*",
"exampleValue" : "bleach, umi-no-kishidan",
"title" : "URL part of desired manga"
},
{
"name" : "Limit",
"identifier" : "limit",
"type" : "number",
"exampleValue" : "10",
"title" : "Number of items to return.\n-1 returns all"
}
]';
}
public function collectData(array $param){
private $request='';
$this->request = '';
$type = "latest"; // can be "latest", "popular" or "path". Default is "latest"!
$path = "latest";
$limit = MANGAREADER_LIMIT;
if(isset($param['category'])){ // Get popular updates
$type = "popular";
$path = "popular";
if($param['category'] !== "all"){
$path .= "/" . $param['category'];
}
}
if(isset($param['path'])){ // Get manga updates
$type = "path";
$path = $param['path'];
}
if(isset($param['limit']) && $param['limit'] !== ""){ // Get manga updates (optional parameter)
$limit = $param['limit'];
}
public function collectData(){
// We'll use the DOM parser for this as it makes navigation easier
$html = file_get_contents("http://www.mangareader.net/" . $path);
$html = $this->getContents($this->getURI());
if(!$html){
$this->returnError('Could not receive data for ' . $path . '!', 400);
$this->returnClientError('Could not receive data for ' . $path . '!');
}
libxml_use_internal_errors(true);
$doc = new DomDocument;
@$doc->loadHTML($html);
libxml_clear_errors();
// Navigate via XPath
$xpath = new DomXPath($doc);
// Build feed based on the context (site updates or manga updates)
if($type === "latest"){
$this->request = '';
switch($this->queriedContext){
case 'Get latest updates':
$this->request = 'Latest updates';
// Query each item (consists of Manga + chapters)
$nodes = $xpath->query("//*[@id='latestchapters']/table//td");
foreach ($nodes as $node){
// Query the manga
$manga = $xpath->query("a[@class='chapter']", $node)->item(0);
// Collect the chapters for each Manga
$chapters = $xpath->query("a[@class='chaptersrec']", $node);
if (isset($manga) && $chapters->length >= 1){
$item = new \Item();
$item->uri = 'http://www.mangareader.net' . htmlspecialchars($manga->getAttribute('href'));
$item->title = htmlspecialchars($manga->nodeValue);
// Add each chapter to the feed
$item->content = "";
$item = array();
$item['uri'] = $this->uri. htmlspecialchars($manga->getAttribute('href'));
$item['title'] = htmlspecialchars($manga->nodeValue);
// Add each chapter to the feed
$item['content'] = "";
foreach ($chapters as $chapter){
if($item->content <> ""){
$item->content .= "<br>";
if($item['content'] <> ""){
$item['content'] .= "<br>";
}
$item->content .= "<a href='http://www.mangareader.net" . htmlspecialchars($chapter->getAttribute('href')) . "'>" . htmlspecialchars($chapter->nodeValue) . "</a>";
$item['content'] .= "<a href='" .$this->uri. htmlspecialchars($chapter->getAttribute('href')) . "'>" . htmlspecialchars($chapter->nodeValue) . "</a>";
}
$this->items[] = $item;
}
}
}
if($type === "popular"){
break;
case 'Get popular mangas':
$pagetitle = $xpath->query(".//*[@id='bodyalt']/h1")->item(0)->nodeValue;
$this->request = substr($pagetitle, 0, strrpos($pagetitle, " -")); // "Popular mangas for ..."
// Query all mangas
$mangas = $xpath->query("//*[@id='mangaresults']/*[@class='mangaresultitem']");
foreach ($mangas as $manga){
foreach ($mangas as $manga){
// The thumbnail is encrypted in a css-style...
// format: "background-image:url('<the part which is actually interesting>')"
$mangaimgelement = $xpath->query(".//*[@class='imgsearchresults']", $manga)->item(0)->getAttribute('style');
$item = new \Item();
$item->title = htmlspecialchars($xpath->query(".//*[@class='manga_name']//a", $manga)->item(0)->nodeValue);
$item->uri = 'http://www.mangareader.net' . $xpath->query(".//*[@class='manga_name']//a", $manga)->item(0)->getAttribute('href');
$item->author = htmlspecialchars($xpath->query("//*[@class='author_name']", $manga)->item(0)->nodeValue);
$item->chaptercount = $xpath->query(".//*[@class='chapter_count']", $manga)->item(0)->nodeValue;
$item->genre = htmlspecialchars($xpath->query(".//*[@class='manga_genre']", $manga)->item(0)->nodeValue);
$item->thumbnailUri = substr($mangaimgelement, 22, strlen($mangaimgelement) - 24);
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" alt="' . $item->title . '" /></a><p>' . $item->genre . '</p><p>' . $item->chaptercount . '</p>';
$mangaimgelement = $xpath->query(".//*[@class='imgsearchresults']", $manga)->item(0)->getAttribute('style');
$thumbnail = substr($mangaimgelement, 22, strlen($mangaimgelement) - 24);
$item = array();
$item['title'] = htmlspecialchars($xpath->query(".//*[@class='manga_name']//a", $manga)->item(0)->nodeValue);
$item['uri'] = $this->uri . $xpath->query(".//*[@class='manga_name']//a", $manga)->item(0)->getAttribute('href');
$item['author'] = htmlspecialchars($xpath->query("//*[@class='author_name']", $manga)->item(0)->nodeValue);
$item['chaptercount'] = $xpath->query(".//*[@class='chapter_count']", $manga)->item(0)->nodeValue;
$item['genre'] = htmlspecialchars($xpath->query(".//*[@class='manga_genre']", $manga)->item(0)->nodeValue);
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $thumbnail . '" alt="' . $item['title'] . '" /></a><p>' . $item['genre'] . '</p><p>' . $item['chaptercount'] . '</p>';
$this->items[] = $item;
}
}
if($type === "path") {
break;
case 'Get manga updates':
$limit = $this->getInput('limit');
if(empty($limit)){
$limit = MANGAREADER_LIMIT;
}
$this->request = $xpath->query(".//*[@id='mangaproperties']//*[@class='aname']")->item(0)->nodeValue;
$query = "(.//*[@id='listing']//tr)[position() > 1]";
if($limit !== -1){
$query = "(.//*[@id='listing']//tr)[position() > 1][position() > last() - " . $limit . "]";
}
}
$chapters = $xpath->query($query);
foreach ($chapters as $chapter){
$item = new \Item();
$item->title = htmlspecialchars($xpath->query("td[1]", $chapter)->item(0)->nodeValue);
$item->uri = 'http://www.mangareader.net' . $xpath->query("td[1]/a", $chapter)->item(0)->getAttribute('href');
$item->timestamp = strtotime($xpath->query("td[2]", $chapter)->item(0)->nodeValue);
$item = array();
$item['title'] = htmlspecialchars($xpath->query("td[1]", $chapter)->item(0)->nodeValue);
$item['uri'] = $this->uri . $xpath->query("td[1]/a", $chapter)->item(0)->getAttribute('href');
$item['timestamp'] = strtotime($xpath->query("td[2]", $chapter)->item(0)->nodeValue);
array_unshift($this->items, $item);
}
}
break;
}
// Return some dummy-data if no content available
if(count($this->items) == 0){
$item = new \Item();
$item->content = "<p>No updates available</p>";
if(empty($this->items)){
$item = array();
$item['content'] = "<p>No updates available</p>";
$this->items[] = $item;
}
}
}
public function getURI(){
switch($this->queriedContext){
case 'Get latest updates':
$path = "latest";
break;
case 'Get popular mangas':
$path = "popular";
if($this->getInput('category') !== "all"){
$path .= "/" . $this->getInput('category');
}
break;
case 'Get manga updates':
$path = $this->getInput('path');
break;
}
return $this->uri . $path;
}
public function getName(){
return (!empty($this->request) ? $this->request . ' - ' : '') . 'Mangareader Bridge';
}
public function getURI(){
return 'http://www.mangareader.net';
}
public function getCacheDuration(){
return 10800; // 3 hours
}
}
?>
?>

58
bridges/MemoLinuxBridge.php
View File

@ -1,58 +0,0 @@
<?php
class MemoLinuxBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "qwertygc";
$this->name = "MemoLinux";
$this->uri = "http://memo-linux.com/";
$this->description = "Returns the 10 newest posts from MemoLinux (full text)";
$this->update = "2015-01-30";
}
public function collectData(array $param){
function StripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function ExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = $html2->find('div.entry-content', 0)->innertext;
$text = preg_replace('@<script[^>]*?>.*?</script>@si', '', $text);
$text = preg_replace('@<div[^>]*?>.*?</div>@si', '', $text);
$text = preg_replace("/<h1.*/", '', $text);
return $text;
}
$html = $this->file_get_html('http://memo-linux.com/feed/') or $this->returnError('Could not request MemoLinux.', 404);
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 10) {
$item = new \Item();
$item->title = StripCDATA($element->find('title', 0)->innertext);
$item->uri = StripCDATA($element->find('guid', 0)->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = ExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'MemoLinux';
}
public function getURI(){
return 'http://memo-linux.com/feed/';
}
public function getCacheDuration(){
return 3600*12; // 12 hours
}
}

70
bridges/MilbooruBridge.php
View File

@ -1,60 +1,38 @@
<?php
class MilbooruBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "mitsukarenai";
$this->name = "Milbooru";
$this->uri = "http://sheslostcontrol.net/moe/shimmie/";
$this->description = "Returns images from given page";
$this->update = "2014-05-25";
public $maintainer = "mitsukarenai";
public $name = "Milbooru";
public $uri = "http://sheslostcontrol.net/moe/shimmie/";
public $description = "Returns images from given page";
$this->parameters[] =
'[
{
"name" : "page",
"identifier" : "p",
"type" : "number"
},
{
"name" : "tags",
"identifier" : "t"
}
]';
}
public function collectData(array $param){
$page = 0;$tags='';
if (isset($param['p'])) {
$page = (int)preg_replace("/[^0-9]/",'', $param['p']);
}
if (isset($param['t'])) {
$tags = urlencode($param['t']);
}
$html = $this->file_get_html("http://sheslostcontrol.net/moe/shimmie/index.php?q=/post/list/$tags/$page") or $this->returnError('Could not request Milbooru.', 404);
public $parameters = array( array(
'p'=>array(
'name'=>'page',
'type'=>'number'
),
't'=>array('name'=>'tags')
));
public function collectData(){
$html = $this->getSimpleHTMLDOM(
$this->uri.'?q=/post/list/'.urlencode($this->getInput('t')).'/'.$this->getInput('p')
)or $this->returnServerError('Could not request Milbooru.');
foreach($html->find('div[class=shm-image-list] span[class=thumb]') as $element) {
$item = new \Item();
$item->uri = 'http://sheslostcontrol.net/moe/shimmie/'.$element->find('a', 0)->href;
$item->postid = (int)preg_replace("/[^0-9]/",'', $element->find('a', 0)->getAttribute('data-post-id'));
$item->timestamp = time();
$item->thumbnailUri = 'http://sheslostcontrol.net/moe/shimmie/'.$element->find('img', 0)->src;
$item->tags = $element->find('a', 0)->getAttribute('data-tags');
$item->title = 'Milbooru | '.$item->postid;
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" /></a><br>Tags: '.$item->tags;
$this->items[] = $item;
$item = array();
$item['uri'] = $this->uri.$element->find('a', 0)->href;
$item['postid'] = (int)preg_replace("/[^0-9]/",'', $element->find('a', 0)->getAttribute('data-post-id'));
$item['timestamp'] = time();
$thumbnailUri = $this->uri.$element->find('img', 0)->src;
$item['tags'] = $element->find('a', 0)->getAttribute('data-tags');
$item['title'] = 'Milbooru | '.$item['postid'];
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $thumbnailUri . '" /></a><br>Tags: '.$item['tags'];
$this->items[] = $item;
}
}
public function getName(){
return 'Milbooru';
}
public function getURI(){
return 'http://sheslostcontrol.net/moe/shimmie/';
}
public function getCacheDuration(){
return 1800; // 30 minutes
}

70
bridges/MondeDiploBridge.php
View File

@ -1,60 +1,26 @@
<?php
class MondeDiploBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "Pitchoule";
public $name = 'Monde Diplomatique';
public $uri = 'http://www.monde-diplomatique.fr/';
public $description = "Returns most recent results from MondeDiplo.";
$this->maintainer = "Pitchoule";
$this->name = "MondeDiplo";
$this->uri = "http://www.monde-diplomatique.fr";
$this->description = "Returns most recent results from MondeDiplo.";
$this->update = "2014-07-22";
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri)
or $this->returnServerError('Could not request MondeDiplo. for : ' . $this->uri);
foreach($html->find('div.unarticle') as $article) {
$element = $article->parent();
$item = array();
$item['uri'] = $this->uri . $element->href;
$item['title'] = $element->find('h3', 0)->plaintext;
$item['content'] = $element->find('div.dates_auteurs', 0)->plaintext . '<br>' . strstr($element->find('div', 0)->plaintext, $element->find('div.dates_auteurs', 0)->plaintext, true);
$this->items[] = $item;
}
}
public function collectData(array $param){
$link = 'http://www.monde-diplomatique.fr';
$html = $this->file_get_html($link) or $this->returnError('Could not request MondeDiplo. for : ' . $link , 404);
foreach($html->find('div.laune') as $element) {
$item = new Item();
$item->uri = 'http://www.monde-diplomatique.fr'.$element->find('a', 0)->href;
$item->title = $element->find('h3', 0)->plaintext;
$item->content = $element->find('div.dates_auteurs', 0)->plaintext. '<br>' .strstr($element->find('div', 0)->plaintext, $element->find('div.dates_auteurs', 0)->plaintext, true);
$this->items[] = $item;
}
$liste = $html->find('div.listes', 0); // First list
foreach ($liste->find('li') as $e) {
$item = new Item();
$item->uri = 'http://www.monde-diplomatique.fr' . $e->find('a', 0)->href;
$item->title = $e->find('a', 0)->plaintext;
$item->content = $e->find('div.dates_auteurs', 0)->plaintext;
$this->items[] = $item;
}
foreach($html->find('div.liste ul li') as $element) {
if ($element->getAttribute('class') != 'intrapub') {
$item = new Item();
$item->uri = 'http://www.monde-diplomatique.fr'.$element->find('a', 0)->href;
$item->title = $element->find('h3', 0)->plaintext;
$item->content = $element->find('div.dates_auteurs', 0)->plaintext . ' <br> ' . $element->find('div.intro', 0)->plaintext;
$this->items[] = $item;
}
}
}
public function getName(){
return 'Monde Diplomatique';
}
public function getURI(){
return 'http://www.monde-diplomatique.fr';
}
public function getCacheDuration(){
return 21600; // 6 hours
}
public function getCacheDuration(){
return 21600; // 6 hours
}
}

64
bridges/MsnMondeBridge.php
View File

@ -1,47 +1,33 @@
<?php
class MsnMondeBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "kranack";
$this->name = "MSN Actu Monde";
$this->uri = "http://www.msn.com/fr-fr/actualite/monde";
$this->description = "Returns the 10 newest posts from MSN Actualités (full text)";
$this->update = "2015-01-30";
}
public function collectData(array $param){
function MsnMondeExtractContent($url, &$item) {
$html2 = $this->file_get_html($url);
$item->content = $html2->find('#content', 0)->find('article', 0)->find('section', 0)->plaintext;
$item->timestamp = strtotime($html2->find('.authorinfo-txt', 0)->find('time', 0)->datetime);
}
$html = $this->file_get_html('http://www.msn.com/fr-fr/actualite/monde') or $this->returnError('Could not request MsnMonde.', 404);
$limit = 0;
foreach($html->find('.smalla') as $article) {
if($limit < 10) {
$item = new \Item();
$item->title = utf8_decode($article->find('h4', 0)->innertext);
$item->uri = "http://www.msn.com" . utf8_decode($article->find('a', 0)->href);
MsnMondeExtractContent($item->uri, $item);
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'MSN Actu Monde';
}
public $maintainer = "kranack";
public $name = 'MSN Actu Monde';
public $uri = 'http://www.msn.com/';
public $description = "Returns the 10 newest posts from MSN Actualités (full text)";
public function getURI(){
return 'http://www.msn.com/fr-fr/actualite/monde';
return $this->uri.'fr-fr/actualite/monde';
}
public function getCacheDuration(){
return 3600; // 1 hour
}
private function MsnMondeExtractContent($url, &$item) {
$html2 = $this->getSimpleHTMLDOM($url);
$item['content'] = $html2->find('#content', 0)->find('article', 0)->find('section', 0)->plaintext;
$item['timestamp'] = strtotime($html2->find('.authorinfo-txt', 0)->find('time', 0)->datetime);
}
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->getURI()) or $this->returnServerError('Could not request MsnMonde.');
$limit = 0;
foreach($html->find('.smalla') as $article) {
if($limit < 10) {
$item = array();
$item['title'] = utf8_decode($article->find('h4', 0)->innertext);
$item['uri'] = $this->uri . utf8_decode($article->find('a', 0)->href);
$this->MsnMondeExtractContent($item['uri'], $item);
$this->items[] = $item;
$limit++;
}
}
}
}

74
bridges/MspabooruBridge.php
View File

@ -1,63 +1,41 @@
<?php
class MspabooruBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "mitsukarenai";
$this->name = "Mspabooru";
$this->uri = "http://mspabooru.com/";
$this->description = "Returns images from given page";
$this->update = "2014-05-25";
public $maintainer = "mitsukarenai";
public $name = "Mspabooru";
public $uri = "http://mspabooru.com/";
public $description = "Returns images from given page";
$this->parameters[] =
'[
{
"name" : "page",
"identifier" : "p",
"type" : "number"
},
{
"name" : "tags",
"identifier" : "t"
}
]';
public $parameters = array( array(
'p'=>array(
'name'=>'page',
'type'=>'number'
),
't'=>array('name'=>'tags')
));
}
public function collectData(array $param){
$page = 0;$tags='';
if (isset($param['p'])) {
$page = (int)preg_replace("/[^0-9]/",'', $param['p']);
$page = $page - 1;
$page = $page * 50;
}
if (isset($param['t'])) {
$tags = urlencode($param['t']);
}
$html = $this->file_get_html("http://mspabooru.com/index.php?page=post&s=list&tags=$tags&pid=$page") or $this->returnError('Could not request Mspabooru.', 404);
public function collectData(){
$html = $this->getSimpleHTMLDOM(
$this->uri.'index.php?page=post&s=list&'
.'&pid='.($this->getInput('p')?($this->getInput('p') -1)*50:'')
.'&tags='.urlencode($this->getInput('t'))
) or $this->returnServerError('Could not request Mspabooru.');
foreach($html->find('div[class=content] span') as $element) {
$item = new \Item();
$item->uri = 'http://mspabooru.com/'.$element->find('a', 0)->href;
$item->postid = (int)preg_replace("/[^0-9]/",'', $element->getAttribute('id'));
$item->timestamp = time();
$item->thumbnailUri = $element->find('img', 0)->src;
$item->tags = $element->find('img', 0)->getAttribute('alt');
$item->title = 'Mspabooru | '.$item->postid;
$item->content = '<a href="' . $item->uri . '"><img src="' . $item->thumbnailUri . '" /></a><br>Tags: '.$item->tags;
$this->items[] = $item;
$item = array();
$item['uri'] = $this->uri.$element->find('a', 0)->href;
$item['postid'] = (int)preg_replace("/[^0-9]/",'', $element->getAttribute('id'));
$item['timestamp'] = time();
$thumbnailUri = $element->find('img', 0)->src;
$item['tags'] = $element->find('img', 0)->getAttribute('alt');
$item['title'] = 'Mspabooru | '.$item['postid'];
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $thumbnailUri . '" /></a><br>Tags: '.$item['tags'];
$this->items[] = $item;
}
}
public function getName(){
return 'Mspabooru';
}
public function getURI(){
return 'http://mspabooru.com/';
}
public function getCacheDuration(){
return 1800; // 30 minutes
}

44
bridges/NakedSecurityBridge.php
View File

@ -1,15 +1,12 @@
<?php
class NakedSecurityBridge extends BridgeAbstract {
public function loadMetadatas() {
$this->maintainer = 'ORelio';
$this->name = $this->getName();
$this->uri = $this->getURI();
$this->description = 'Returns the newest articles.';
$this->update = '2016-04-30';
}
public $maintainer = 'ORelio';
public $name = 'Naked Security';
public $uri = 'https://nakedsecurity.sophos.com/';
public $description = 'Returns the newest articles.';
public function collectData(array $param) {
public function collectData(){
function StripRecursiveHTMLSection($string, $tag_name, $tag_start) {
$open_tag = '<'.$tag_name;
@ -36,7 +33,7 @@ class NakedSecurityBridge extends BridgeAbstract {
}
$feedUrl = 'https://feeds.feedburner.com/nakedsecurity?format=xml';
$html = $this->file_get_html($feedUrl) or $this->returnError('Could not request '.$this->getName().': '.$feedUrl, 500);
$html = $this->getSimpleHTMLDOM($feedUrl) or $this->returnServerError('Could not request '.$this->getName().': '.$feedUrl);
$limit = 0;
foreach ($html->find('item') as $element) {
@ -44,7 +41,7 @@ class NakedSecurityBridge extends BridgeAbstract {
//Retrieve article Uri and get that page
$article_uri = $element->find('guid', 0)->plaintext;
$article_html = $this->file_get_html($article_uri) or $this->returnError('Could not request '.$this->getName().': '.$article_uri, 500);
$article_html = $this->getSimpleHTMLDOM($article_uri) or $this->returnServerError('Could not request '.$this->getName().': '.$article_uri);
//Build article contents from corresponding elements
$article_title = trim($element->find('title', 0)->plaintext);
@ -57,28 +54,15 @@ class NakedSecurityBridge extends BridgeAbstract {
$article_content = '<p><img src="'.$article_image.'" /></p><p><b>'.$article_summary.'</b></p>'.$article_content;
//Build and add final item
$item = new \Item();
$item->uri = $article_uri;
$item->title = $article_title;
$item->thumbnailUri = $article_image;
$item->author = $article_html->find('a[rel=author]', 0)->plaintext;
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = $article_content;
$item = array();
$item['uri'] = $article_uri;
$item['title'] = $article_title;
$item['author'] = $article_html->find('a[rel=author]', 0)->plaintext;
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $article_content;
$this->items[] = $item;
$limit++;
}
}
}
public function getName() {
return 'Naked Security';
}
public function getURI() {
return 'https://nakedsecurity.sophos.com/';
}
public function getCacheDuration() {
return 3600; //1 hour
}
}
}

39
bridges/NasaApodBridge.php
View File

@ -1,31 +1,26 @@
<?php
class NasaApodBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "corenting";
public $name = "NASA APOD Bridge";
public $uri = "http://apod.nasa.gov/apod/";
public $description = "Returns the 3 latest NASA APOD pictures and explanations";
$this->maintainer = "corenting";
$this->name = "NASA APOD Bridge";
$this->uri = "http://apod.nasa.gov/apod/astropix.html";
$this->description = "Returns the 3 latest NASA APOD pictures and explanations";
$this->update = "2014-08-27";
public function collectData(){
}
public function collectData(array $param) {
$html = $this->file_get_html('http://apod.nasa.gov/apod/archivepix.html') or $this->returnError('Error while downloading the website content', 404);
$html = $this->getSimpleHTMLDOM($this->uri.'archivepix.html') or $this->returnServerError('Error while downloading the website content');
$list = explode("<br>", $html->find('b', 0)->innertext);
for($i = 0; $i < 3;$i++)
{
$line = $list[$i];
$item = new \Item();
$item = array();
$uri_page = $html->find('a',$i + 3)->href;
$uri = 'http://apod.nasa.gov/apod/'.$uri_page;
$item->uri = $uri;
$uri = $this->uri.$uri_page;
$item['uri'] = $uri;
$picture_html = $this->file_get_html($uri);
$picture_html = $this->getSimpleHTMLDOM($uri);
$picture_html_string = $picture_html->innertext;
//Extract image and explanation
@ -36,23 +31,15 @@ class NasaApodBridge extends BridgeAbstract{
//Extract date from the picture page
$date = explode(" ", $picture_html->find('p',1)->innertext);
$item->timestamp = strtotime($date[4].$date[3].$date[2]);
$item['timestamp'] = strtotime($date[4].$date[3].$date[2]);
//Other informations
$item->content = $media.'<br />'.$explanation;
$item->title = $picture_html->find('b',0)->innertext;
$item['content'] = $media.'<br />'.$explanation;
$item['title'] = $picture_html->find('b',0)->innertext;
$this->items[] = $item;
}
}
public function getName(){
return 'NASA APOD';
}
public function getURI(){
return 'http://apod.nasa.gov/apod/astropix.html';
}
public function getCacheDuration(){
return 3600*12; // 12 hours
}

50
bridges/NeuviemeArtBridge.php
View File

@ -1,15 +1,12 @@
<?php
class NeuviemeArtBridge extends BridgeAbstract {
public function loadMetadatas() {
$this->maintainer = "ORelio";
$this->name = "9ème Art";
$this->uri = "http://www.9emeart.fr/";
$this->description = "Returns the newest articles.";
$this->update = "2016-02-05";
}
public $maintainer = "ORelio";
public $name = '9ème Art Bridge';
public $uri = "http://www.9emeart.fr/";
public $description = "Returns the newest articles.";
public function collectData(array $param) {
public function collectData(){
function StripWithDelimiters($string, $start, $end) {
while (strpos($string, $start) !== false) {
@ -19,8 +16,8 @@ class NeuviemeArtBridge extends BridgeAbstract {
} return $string;
}
$feedUrl = 'http://www.9emeart.fr/9emeart.rss';
$html = $this->file_get_html($feedUrl) or $this->returnError('Could not request 9eme Art: '.$feedUrl, 500);
$feedUrl = $this->uri.'9emeart.rss';
$html = $this->getSimpleHTMLDOM($feedUrl) or $this->returnServerError('Could not request 9eme Art: '.$feedUrl);
$limit = 0;
foreach ($html->find('item') as $element) {
@ -28,43 +25,30 @@ class NeuviemeArtBridge extends BridgeAbstract {
//Retrieve article Uri and get that page
$article_uri = $element->find('guid', 0)->plaintext;
$article_html = $this->file_get_html($article_uri) or $this->returnError('Could not request 9eme Art: '.$article_uri, 500);
$article_html = $this->getSimpleHTMLDOM($article_uri) or $this->returnServerError('Could not request 9eme Art: '.$article_uri);
//Build article contents from corresponding elements
$article_title = trim($element->find('title', 0)->plaintext);
$article_image = $element->find('enclosure', 0)->url;
foreach ($article_html->find('img.img_full') as $img)
if ($img->alt == $article_title)
$article_image = 'http://www.9emeart.fr'.$img->src;
$article_image = $this->uri.$img->src;
$article_content = '<p><img src="'.$article_image.'" /></p>'
.str_replace('src="/', 'src="http://www.9emeart.fr/', $article_html->find('div.newsGenerique_con', 0)->innertext);
.str_replace('src="/', 'src="'.$this->uri, $article_html->find('div.newsGenerique_con', 0)->innertext);
$article_content = StripWithDelimiters($article_content, '<script', '</script>');
$article_content = StripWithDelimiters($article_content, '<style', '</style>');
$article_content = StripWithDelimiters($article_content, '<link', '>');
//Build and add final item
$item = new \Item();
$item->uri = $article_uri;
$item->title = $article_title;
$item->thumbnailUri = $element->find('enclosure', 0)->url;
$item->author = $article_html->find('a[class=upp transition_fast upp]', 0)->plaintext;
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = $article_content;
$item = array();
$item['uri'] = $article_uri;
$item['title'] = $article_title;
$item['author'] = $article_html->find('a[class=upp transition_fast upp]', 0)->plaintext;
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $article_content;
$this->items[] = $item;
$limit++;
}
}
}
public function getName() {
return '9ème Art Bridge';
}
public function getURI() {
return 'http://www.9emeart.fr/';
}
public function getCacheDuration() {
return 3600; //1 hour
}
}
}

73
bridges/NextInpactBridge.php
View File

@ -1,64 +1,43 @@
<?php
class NextInpactBridge extends BridgeAbstract {
public function loadMetadatas() {
$this->maintainer = "qwertygc";
$this->name = "NextInpact Bridge";
$this->uri = "http://www.nextinpact.com/";
$this->description = "Returns the newest articles.";
$this->update = "2015-10-23";
public $maintainer = "qwertygc";
public $name = "NextInpact Bridge";
public $uri = "http://www.nextinpact.com/";
public $description = "Returns the newest articles.";
private function StripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
public function collectData(array $param) {
private function ExtractContent($url) {
$html2 = $this->getSimpleHTMLDOM($url);
$text = '<p><em>'.$html2->find('span.sub_title', 0)->innertext.'</em></p>'
.'<p><img src="'.$html2->find('div.container_main_image_article', 0)->find('img.dedicated',0)->src.'" alt="-" /></p>'
.'<div>'.$html2->find('div[itemprop=articleBody]', 0)->innertext.'</div>';
$premium_article = $html2->find('h2.title_reserve_article', 0);
if (is_object($premium_article))
$text = $text.'<p><em>'.$premium_article->innertext.'</em></p>';
return $text;
}
function StripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function ExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = '<p><em>'.$html2->find('span.sub_title', 0)->innertext.'</em></p>'
.'<p><img src="'.$html2->find('div.container_main_image_article', 0)->find('img.dedicated',0)->src.'" alt="-" /></p>'
.'<div>'.$html2->find('div[itemprop=articleBody]', 0)->innertext.'</div>';
$premium_article = $html2->find('h2.title_reserve_article', 0);
if (is_object($premium_article))
$text = $text.'<p><em>'.$premium_article->innertext.'</em></p>';
return $text;
}
$html = $this->file_get_html('http://www.nextinpact.com/rss/news.xml') or $this->returnError('Could not request NextInpact.', 404);
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri.'rss/news.xml') or $this->returnServerError('Could not request NextInpact.');
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 3) {
$item = new \Item();
$item->title = StripCDATA($element->find('title', 0)->innertext);
$item->uri = StripCDATA($element->find('guid', 0)->plaintext);
$item->thumbnailUri = StripCDATA($element->find('enclosure', 0)->url);
$item->author = StripCDATA($element->find('author', 0)->innertext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = ExtractContent($item->uri);
$item = array();
$item['title'] = $this->StripCDATA($element->find('title', 0)->innertext);
$item['uri'] = $this->StripCDATA($element->find('guid', 0)->plaintext);
$item['author'] = $this->StripCDATA($element->find('creator', 0)->innertext);
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $this->ExtractContent($item['uri']);
$this->items[] = $item;
$limit++;
}
}
}
public function getName() {
return 'Nextinpact Bridge';
}
public function getURI() {
return 'http://www.nextinpact.com/';
}
public function getCacheDuration() {
return 3600; // 1 hour
// return 0;
}
}

134
bridges/NextgovBridge.php
View File

@ -1,39 +1,31 @@
<?php
class NextgovBridge extends BridgeAbstract {
public function loadMetadatas() {
public $maintainer = 'ORelio';
public $name = 'Nextgov Bridge';
public $uri = 'https://www.nextgov.com/';
public $description = 'USA Federal technology news, best practices, and web 2.0 tools.';
$this->maintainer = 'ORelio';
$this->name = $this->getName();
$this->uri = $this->getURI();
$this->description = 'USA Federal technology news, best practices, and web 2.0 tools.';
$this->update = '2016-07-20';
public $parameters = array( array(
'category'=>array(
'name'=>'Category',
'type'=>'list',
'values'=>array(
'All'=>'all',
'Technology News'=>'technology-news',
'CIO Briefing'=>'cio-briefing',
'Emerging Tech'=>'emerging-tech',
'Cloud'=>'cloud-computing',
'Cybersecurity'=>'cybersecurity',
'Mobile'=>'mobile',
'Health'=>'health',
'Defense'=>'defense',
'Big Data'=>'big-data'
)
)
));
$this->parameters[] =
'[
{
"name" : "Category",
"type" : "list",
"identifier" : "category",
"values" :
[
{ "name" : "All", "value" : "all" },
{ "name" : "Technology News", "value" : "technology-news" },
{ "name" : "CIO Briefing", "value" : "cio-briefing" },
{ "name" : "Emerging Tech", "value" : "emerging-tech" },
{ "name" : "Cloud", "value" : "cloud-computing" },
{ "name" : "Cybersecurity", "value" : "cybersecurity" },
{ "name" : "Mobile", "value" : "mobile" },
{ "name" : "Health", "value" : "health" },
{ "name" : "Defense", "value" : "defense" },
{ "name" : "Big Data", "value" : "big-data" }
]
}
]';
}
public function collectData(array $param) {
public function collectData(){
function ExtractFromDelimiters($string, $start, $end) {
if (strpos($string, $start) !== false) {
@ -51,60 +43,40 @@ class NextgovBridge extends BridgeAbstract {
} return $string;
}
$category = $param['category'];
if (empty($category))
$category = 'all';
if ($category !== preg_replace('/[^a-z-]+/', '', $category) || strlen($category > 32))
$this->returnError('Invalid "category" parameter.', 400);
$category = $this->getInput('category');
$url = $this->getURI().'rss/'.$category.'/';
$html = $this->file_get_html($url) or $this->returnError('Could not request Nextgov: '.$url, 500);
$html = $this->getSimpleHTMLDOM($url) or $this->returnServerError('Could not request Nextgov: '.$url);
$limit = 0;
foreach ($html->find('item') as $element) {
if ($limit < 10) {
$article_url = ExtractFromDelimiters($element->innertext, '<link>', '</link>');
$article_author = ExtractFromDelimiters($element->innertext, 'dc/elements/1.1/">', '</dc:creator>');
$article_title = $element->find('title', 0)->plaintext;
$article_subtitle = $element->find('description', 0)->plaintext;
$article_timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$article_thumbnail = ExtractFromDelimiters($element->innertext, '<media:content url="', '"');
$article = $this->file_get_html($article_url) or $this->returnError('Could not request Nextgov: '.$article_url, 500);
$contents = $article->find('div.wysiwyg', 0)->innertext;
$contents = StripWithDelimiters($contents, '<div class="ad-container">', '</div>');
$contents = StripWithDelimiters($contents, '<div', '</div>'); //ad outer div
$contents = StripWithDelimiters($contents, '<script', '</script>');
$contents = ($article_thumbnail == '' ? '' : '<p><img src="'.$article_thumbnail.'" /></p>')
.'<p><b>'.$article_subtitle.'</b></p>'
.trim($contents);
if ($article_thumbnail == '')
$article_thumbnail = 'http://cdn.nextgov.com/nextgov/images/logo.png';
$item = new \Item();
$item->uri = $article_url;
$item->title = $article_title;
$item->author = $article_author;
$item->thumbnailUri = $article_thumbnail;
$item->timestamp = $article_timestamp;
$item->content = $contents;
$this->items[] = $item;
$limit++;
if ($limit >= 10) {
break;
}
$article_url = ExtractFromDelimiters($element->innertext, '<link>', '</link>');
$article_author = ExtractFromDelimiters($element->innertext, 'dc/elements/1.1/">', '</dc:creator>');
$article_title = $element->find('title', 0)->plaintext;
$article_subtitle = $element->find('description', 0)->plaintext;
$article_timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$article_thumbnail = ExtractFromDelimiters($element->innertext, '<media:content url="', '"');
$article = $this->getSimpleHTMLDOM($article_url) or $this->returnServerError('Could not request Nextgov: '.$article_url);
$contents = $article->find('div.wysiwyg', 0)->innertext;
$contents = StripWithDelimiters($contents, '<div class="ad-container">', '</div>');
$contents = StripWithDelimiters($contents, '<div', '</div>'); //ad outer div
$contents = StripWithDelimiters($contents, '<script', '</script>');
$contents = ($article_thumbnail == '' ? '' : '<p><img src="'.$article_thumbnail.'" /></p>')
.'<p><b>'.$article_subtitle.'</b></p>'
.trim($contents);
$item = array();
$item['uri'] = $article_url;
$item['title'] = $article_title;
$item['author'] = $article_author;
$item['timestamp'] = $article_timestamp;
$item['content'] = $contents;
$this->items[] = $item;
$limit++;
}
}
public function getName() {
return 'Nextgov Bridge';
}
public function getURI() {
return 'https://www.nextgov.com/';
}
public function getCacheDuration() {
return 3600; //1 hour
}
}
}

89
bridges/NiceMatinBridge.php
View File

@ -1,62 +1,47 @@
<?php
class NiceMatinBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "pit-fgfjiudghdf";
public $name = "NiceMatin";
public $uri = "http://www.nicematin.com/";
public $description = "Returns the 10 newest posts from NiceMatin (full text)";
$this->maintainer = "pit-fgfjiudghdf";
$this->name = "NiceMatin";
$this->uri = "http://www.nicematin.com/";
$this->description = "Returns the 10 newest posts from NiceMatin (full text)";
$this->update = "2014-05-26";
private function NiceMatinExtractContent($url) {
$html = $this->getSimpleHTMLDOM($url);
if(!$html)
return 'Could not acquire content from url: ' . $url . '!';
$content = $html->find('article', 0);
if(!$content)
return 'Could not find \'section\'!';
$text = preg_replace('#<script(.*?)>(.*?)</script>#is', '', $content->innertext);
$text = strip_tags($text, '<p><a><img>');
return $text;
}
public function collectData(array $param){
public function collectData(){
$html = $this->getSimpleHTMLDOM($this->uri.'derniere-minute/rss')
or $this->returnServerError('Could not request NiceMatin.');
$limit = 0;
function NiceMatinUrl($string) {
$string = str_replace('</link>', '', $string);
//$string = str_replace('.+', '', $string);
$string = preg_replace('/html.*http.*/i','html',$string);
$string = preg_replace('/.*http/i','http',$string);
return $string;
}
foreach($html->find('item') as $element) {
if($limit >= 10) {
break;
}
// We need to fix the 'link' tag as simplehtmldom cannot parse it (just rename it and load back as dom)
$element_text = $element->outertext;
$element_text = str_replace('<link>', '<url>', $element_text);
$element_text = str_replace('</link>', '</url>', $element_text);
$element = str_get_html($element_text);
function NiceMatinExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = $html2->find('figure[itemprop=associatedMedia]', 0)->innertext;
$text .= $html2->find('div[id=content-article]', 0)->innertext;
return $text;
}
$html = $this->file_get_html('http://www.nicematin.com/derniere-minute/rss') or $this->returnError('Could not request NiceMatin.', 404);
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 10) {
$item = new \Item();
//$item->title = NiceMatinStripCDATA($element->find('title', 0)->innertext);
$item->title = $element->find('title', 0)->innertext;
$item->uri = NiceMatinUrl($element->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = NiceMatinExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'NiceMatin';
}
public function getURI(){
return 'http://www.nicematin.com/';
}
public function getCacheDuration(){
return 3600; // 1 hour
}
$item = array();
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = $element->find('url', 0)->innertext;
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$item['content'] = $this->NiceMatinExtractContent($item['uri']);
$this->items[] = $item;
$limit++;
}
}
}

65
bridges/NovelUpdatesBridge.php
View File

@ -1,33 +1,30 @@
<?php
class NovelUpdatesBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "albirew";
public $name = "Novel Updates";
public $uri = "http://www.novelupdates.com/";
public $description = "Returns releases from Novel Updates";
public $parameters = array( array(
'n'=>array(
'name'=>'Novel URL',
'patterns'=>'http:\/\/www.novelupdates.com\/.*',
'required'=>true
)
));
$this->maintainer = "albirew";
$this->name = "Novel Updates";
$this->uri = "http://www.novelupdates.com/";
$this->description = "Returns releases from Novel Updates";
$this->update = "2016-05-21";
$this->parameters[] =
'[
{
"name" : "Novel URL",
"identifier" : "n"
}
]';
}
private $seriesTitle='';
public function collectData(array $param){
if (!isset($param['n']))
$this->returnError('You must specify the novel URL (/series/...)', 400);
$thread = parse_url($param['n']) or $this->returnError('This URL seems malformed, please check it.', 400);
public function collectData(){
$thread = parse_url($this->getInput('n'))
or $this->returnClientError('This URL seems malformed, please check it.');
if($thread['host'] !== 'www.novelupdates.com')
$this->returnError('NovelUpdates URL only.', 400);
$this->returnClientError('NovelUpdates URL only.');
if(strpos($thread['path'], 'series/') === FALSE)
$this->returnError('You must specify the novel URL.', 400);
$url = 'http://www.novelupdates.com'.$thread['path'].'';
$fullhtml = $this->file_get_html($url) or $this->returnError("Could not request NovelUpdates, novel not found", 404);
$this->request = $fullhtml->find('h4.seriestitle', 0)->plaintext;
$this->returnClientError('You must specify the novel URL.');
$url = $this->uri.$thread['path'].'';
$fullhtml = $this->getSimpleHTMLDOM($url) or $this->returnServerError("Could not request NovelUpdates, novel not found");
$this->seriesTitle = $fullhtml->find('h4.seriestitle', 0)->plaintext;
// dirty fix for nasty simpledom bug: https://github.com/sebsauvage/rss-bridge/issues/259
// forcefully removes tbody
$html = $fullhtml->find('table#myTable', 0)->innertext;
@ -35,24 +32,20 @@ class NovelUpdatesBridge extends BridgeAbstract{
$html = stristr($html, '<tr>'); //remove tbody
$html = str_get_html(stristr($html, '</tbody>', true)); //remove last tbody and get back as an array
foreach($html->find('tr') as $element){
$item = new \Item();
$item->uri = $element->find('td', 2)->find('a', 0)->href;
$item->title = $element->find('td', 2)->find('a', 0)->plaintext;
$item->team = $element->find('td', 1)->innertext;
$item->timestamp = strtotime($element->find('td', 0)->plaintext);
$item->content = '<a href="'.$item->uri.'">'.$this->request.' - '.$item->title.'</a> by '.$item->team.'<br><a href="'.$item->uri.'">'.$fullhtml->find('div.seriesimg', 0)->innertext.'</a>';
$item = array();
$item['uri'] = $element->find('td', 2)->find('a', 0)->href;
$item['title'] = $element->find('td', 2)->find('a', 0)->plaintext;
$item['team'] = $element->find('td', 1)->innertext;
$item['timestamp'] = strtotime($element->find('td', 0)->plaintext);
$item['content'] = '<a href="'.$item['uri'].'">'.$this->seriesTitle.' - '.$item['title'].'</a> by '.$item['team'].'<br><a href="'.$item['uri'].'">'.$fullhtml->find('div.seriesimg', 0)->innertext.'</a>';
$this->items[] = $item;
}
}
public function getName(){
return (!empty($this->request) ? $this->request.' - ' : '') .'Novel Updates';
}
public function getURI(){
return 'http://www.novelupdates.com/';
}
public function getDescription(){
return "Novel Updates - Directory of Translated Novels";
return (!empty($this->seriesTitle) ? $this->seriesTitle.' - ' : '') .'Novel Updates';
}
public function getCacheDuration(){
return 21600; // 6 hours
}

47
bridges/NumeramaBridge.php
View File

@ -1,17 +1,12 @@
<?php
class NumeramaBridge extends BridgeAbstract{
class NumeramaBridge extends HttpCachingBridgeAbstract {
public function loadMetadatas() {
public $maintainer = 'mitsukarenai';
public $name = 'Numerama';
public $uri = 'http://www.numerama.com/';
public $description = 'Returns the 5 newest posts from Numerama (full text)';
$this->maintainer = 'mitsukarenai';
$this->name = 'Numerama';
$this->uri = 'http://www.numerama.com/';
$this->description = 'Returns the 5 newest posts from Numerama (full text)';
$this->update = '2016-07-19';
}
public function collectData(array $param) {
public function collectData(){
function NumeramaStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
@ -19,38 +14,32 @@ class NumeramaBridge extends BridgeAbstract{
return $string;
}
$feed = $this->getURI().'feed/';
$html = $this->file_get_html($feed) or $this->returnError('Could not request Numerama: '.$feed, 500);
$feed = $this->uri.'feed/';
$html = $this->getSimpleHTMLDOM($feed) or $this->returnServerError('Could not request Numerama: '.$feed);
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 5) {
$item = new \Item();
$item->title = html_entity_decode(NumeramaStripCDATA($element->find('title', 0)->innertext));
$item->author = NumeramaStripCDATA($element->find('dc:creator', 0)->innertext);
$item->uri = NumeramaStripCDATA($element->find('guid', 0)->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item = array();
$item['title'] = html_entity_decode(NumeramaStripCDATA($element->find('title', 0)->innertext));
$item['author'] = NumeramaStripCDATA($element->find('dc:creator', 0)->innertext);
$item['uri'] = NumeramaStripCDATA($element->find('guid', 0)->plaintext);
$item['timestamp'] = strtotime($element->find('pubDate', 0)->plaintext);
$article_url = NumeramaStripCDATA($element->find('guid', 0)->plaintext);
$article_html = $this->file_get_html($article_url) or $this->returnError('Could not request Numerama: '.$article_url, 500);
if($this->get_cached_time($article_url) <= strtotime('-24 hours'))
$this->remove_from_cache($article_url);
$article_html = $this->get_cached($article_url) or $this->returnServerError('Could not request Numerama: '.$article_url);
$contents = $article_html->find('section[class=related-article]', 0)->innertext = ''; // remove related articles block
$contents = '<img alt="" style="max-width:300px;" src="'.$article_html->find('meta[property=og:image]', 0)->getAttribute('content').'">'; // add post picture
$contents = $contents.$article_html->find('article[class=post-content]', 0)->innertext; // extract the post
$item->content = $contents;
$item['content'] = $contents;
$this->items[] = $item;
$limit++;
}
}
}
public function getName() {
return 'Numerama';
}
public function getURI() {
return 'http://www.numerama.com/';
}
public function getCacheDuration() {

106
bridges/OpenClassroomsBridge.php
View File

@ -1,92 +1,50 @@
<?php
class OpenClassroomsBridge extends BridgeAbstract{
public function loadMetadatas() {
public $maintainer = "sebsauvage";
public $name = "OpenClassrooms Bridge";
public $uri = "https://openclassrooms.com/";
public $description = "Returns latest tutorials from OpenClassrooms.";
$this->maintainer = "sebsauvage";
$this->name = "OpenClassrooms Bridge";
$this->uri = "https://openclassrooms.com/";
$this->description = "Returns latest tutorials from OpenClassrooms.";
$this->update = "2015-10-30";
public $parameters = array( array(
'u'=>array(
'name'=>'Catégorie',
'type'=>'list',
'values'=>array(
'Arts & Culture'=>'arts',
'Code'=>'code',
'Design'=>'design',
'Entreprise'=>'business',
'Numérique'=>'digital',
'Sciences'=>'sciences',
'Sciences Humaines'=>'humainities',
'Systèmes d\'information'=>'it',
'Autres'=>'others'
)
)
));
$this->parameters[] =
'[
{
"name" : "Catégorie",
"identifier" : "u",
"type" : "list",
"values" : [
{
"name" : "Arts & Culture",
"value" : "arts"
},
{
"name" : "Code",
"value" : "code"
},
{
"name" : "Design",
"value" : "design"
},
{
"name" : "Entreprise",
"value" : "business"
},
{
"name" : "Numérique",
"value" : "digital"
},
{
"name" : "Sciences",
"value" : "sciences"
},
{
"name" : "Sciences Humaines",
"value" : "humainities"
},
{
"name" : "Systèmes d\'information",
"value" : "it"
},
{
"name" : "Autres",
"value" : "others"
}
]
}
]';
}
public function collectData(array $param){
if (empty($param['u']))
public function collectData(){
if (empty($this->getInput('u')))
{
$this->returnError('Error: You must chose a category.', 404);
$this->returnServerError('Error: You must chose a category.');
}
$html = '';
$link = 'https://openclassrooms.com/courses?categories='.$param['u'].'&title=&sort=updatedAt+desc';
$html = $this->file_get_html($link) or $this->returnError('Could not request OpenClassrooms.', 404);
$html = '';
$link = 'https://openclassrooms.com/courses?categories='.$this->getInput('u').'&title=&sort=updatedAt+desc';
$html = $this->getSimpleHTMLDOM($link) or $this->returnServerError('Could not request OpenClassrooms.');
foreach($html->find('.courseListItem') as $element) {
$item = new \Item();
$item->uri = 'https://openclassrooms.com'.$element->find('a', 0)->href;
$item->title = $element->find('h3', 0)->plaintext;
$item->content = $element->find('slidingItem__descriptionContent', 0)->plaintext;
$item = array();
$item['uri'] = 'https://openclassrooms.com'.$element->find('a', 0)->href;
$item['title'] = $element->find('h3', 0)->plaintext;
$item['content'] = $element->find('slidingItem__descriptionContent', 0)->plaintext;
$this->items[] = $item;
}
}
public function getName(){
return 'OpenClassrooms';
}
public function getURI(){
return 'https://openclassrooms.com/';
}
public function getCacheDuration(){
return 21600; // 6 hours
}

56
bridges/OpenTheoryBridge.php
View File

@ -1,56 +0,0 @@
<?php
class OpenTheoryBridge extends BridgeAbstract{
public function loadMetadatas() {
$this->maintainer = "qwertygc";
$this->name = "Opentheory";
$this->uri = "http://open1theory.com";
$this->description = "Returns the 5 newest posts from OpenTheory (full text)";
$this->update = "02-08-2014";
}
public function collectData(array $param){
function StripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function ExtractContent($url) {
$html2 = $this->file_get_html($url);
$text = $html2->find('div.entry-content', 0)->innertext;
$text = preg_replace('@<script[^>]*?>.*?</script>@si', '', $text);
return $text;
}
$html = $this->file_get_html('http://open1theory.com/feed') or $this->returnError('Could not request OpenTheory.', 404);
$limit = 0;
foreach($html->find('item') as $element) {
if($limit < 5) {
$item = new \Item();
$item->title = StripCDATA($element->find('title', 0)->innertext);
$item->uri = StripCDATA($element->find('guid', 0)->plaintext);
$item->timestamp = strtotime($element->find('pubDate', 0)->plaintext);
$item->content = ExtractContent($item->uri);
$this->items[] = $item;
$limit++;
}
}
}
public function getName(){
return 'OpenTheory';
}
public function getURI(){
return 'http://open1theory.com/feed';
}
public function getCacheDuration(){
return 3600; // 1 hour
// return 0; // 1 hour
}
}

123
bridges/ParuVenduImmoBridge.php
View File

@ -1,109 +1,96 @@
<?php
class ParuVenduImmoBridge extends BridgeAbstract
{
private $request = '';
public function loadMetadatas() {
$this->maintainer = "polo2ro";
$this->name = "Paru Vendu Immobilier";
$this->uri = "http://www.paruvendu.fr/immobilier/";
$this->description = "Returns the ads from the first page of search result.";
$this->update = "2015-02-02";
public $maintainer = "polo2ro";
public $name = "Paru Vendu Immobilier";
public $uri = "http://www.paruvendu.fr";
public $description = "Returns the ads from the first page of search result.";
$this->parameters[] =
'[
{
"name": "Minimal surface m²",
"type" : "number",
"identifier" : "minarea"
},
{
"name" : "Max price",
"type" : "number",
"identifier" : "maxprice"
},
{
"name" : "Country code",
"type" : "text",
"identifier" : "pa",
"exampleValue" : "FR"
},
{
"name" : "department numbers or postal codes, comma-separated",
"type" : "text",
"identifier" : "lo"
}
public $parameters = array( array(
'minarea'=>array(
'name'=>'Minimal surface m²',
'type'=>'number'
),
'maxprice'=>array(
'name'=>'Max price',
'type'=>'number'
),
'pa'=>array(
'name'=>'Country code',
'exampleValue'=>'FR'
),
'lo'=>array('name'=>'department numbers or postal codes, comma-separated')
));
]';
}
public function collectData(array $param)
public function collectData()
{
$html = '';
$num = 20;
$appartment = '&tbApp=1&tbDup=1&tbChb=1&tbLof=1&tbAtl=1&tbPla=1';
$maison = '&tbMai=1&tbVil=1&tbCha=1&tbPro=1&tbHot=1&tbMou=1&tbFer=1';
$link = $this->getURI().'/immobilier/annonceimmofo/liste/listeAnnonces?tt=1'.$appartment.$maison;
if (isset($param['minarea'])) {
$this->request .= ' '.$param['minarea'].' m2';
$link .= '&sur0='.urlencode($param['minarea']);
$link = $this->uri.'/immobilier/annonceimmofo/liste/listeAnnonces?tt=1'.$appartment.$maison;
if ($this->getInput('minarea')) {
$link .= '&sur0='.urlencode($this->getInput('minarea'));
}
if (isset($param['maxprice'])) {
$link .= '&px1='.urlencode($param['maxprice']);
}
if (isset($param['pa'])) {
$link .= '&pa='.urlencode($param['pa']);
}
if (isset($param['lo'])) {
$this->request .= ' In: '.$param['lo'];
$link .= '&lo='.urlencode($param['lo']);
if ($this->getInput('maxprice')) {
$link .= '&px1='.urlencode($this->getInput('maxprice'));
}
$html = $this->file_get_html($link) or $this->returnError('Could not request paruvendu.', 404);
if ($this->getInput('pa')) {
$link .= '&pa='.urlencode($this->getInput('pa'));
}
if ($this->getInput('lo')) {
$link .= '&lo='.urlencode($this->getInput('lo'));
}
$html = $this->getSimpleHTMLDOM($link) or $this->returnServerError('Could not request paruvendu.');
foreach($html->find('div.annonce a') as $element) {
if (!$element->title) {
continue;
}
$img ='';
foreach($element->find('span.img img') as $img) {
if ($img->original) {
$img = '<img src="'.$img->original.'" />';
}
}
$desc = $element->find('span.desc')[0]->innertext;
$desc = str_replace("voir l'annonce", '', $desc);
$price = $element->find('span.price')[0]->innertext;
list($href) = explode('#', $element->href);
$item = new \Item();
$item->uri = $this->getURI().$href;
$item->title = $element->title;
$item->content = $img.$desc.$price;
$item = array();
$item['uri'] = $this->uri.$href;
$item['title'] = $element->title;
$item['content'] = $img.$desc.$price;
$this->items[] = $item;
}
}
public function getName(){
return 'Paru Vendu Immobilier'.$this->request;
}
public function getURI(){
return 'http://www.paruvendu.fr';
$request='';
if($this->getInput('minarea') &&
!empty($this->getInput('minarea'))
){
$request .= ' '.$this->getInput('minarea').' m2';
}
if($this->getInput('lo') &&
!empty($this->getInput('lo'))){
$request .= ' In: '.$this->getInput('lo');
}
return 'Paru Vendu Immobilier'.$request;
}
public function getCacheDuration(){

Some files were not shown because too many files have changed in this diff Show More