Compare commits

...

43 Commits

Author SHA1 Message Date
July
d6a9da1cc8
[SubstackProfileBridge] Add new bridge (#4507) 2025-04-03 07:51:58 +02:00
Jonathan Kay
85962e18d3
[GoComicsBridge] New layout fix and added features (#4510)
* Updated to use the new layout launched April 1st
* Adds new title date/full name option
* Adds limit option for how many days of comics to get
2025-04-03 07:50:16 +02:00
July
a19b63e840
[AO3Bridge] Add option to make one entry per fic (#4508) 2025-04-02 04:09:28 +02:00
tillcash
5365b57638
[MinecraftBridge] fix favicon (#4506) 2025-04-02 03:57:40 +02:00
Dag
462c005f2c
fix: dont read /etc if open_basedir #4502 (#4505) 2025-04-01 01:15:59 +02:00
ORelio
db42f2786c
[FeedExpander] Add prepareXml() overridable function (#4485)
* FeedExpander: Remove tailing content in XML

- Move preprocessing code into overridable preprocessXml()
- Auto-remove trailing data after root xml node

* FeedExpander: Add PR reference with use case

* FeedExpander: Code linting

* [FeedExpander] Keep content at end of document for now

Will add back later if more sites have the same issue

* [FeedExpander] prepareXml: Add type hints
2025-04-01 00:42:08 +02:00
ORelio
26a4c255d3
[html] convertLazyLoading: Add parseSrcset() (#4503)
* [html] convertLazyLoading: Add parseSrcset()

Add srcset parser closer to the specifications

* [html] code linting

* [html] parseSrcset: Add type hints, check preg_match_all
2025-04-01 00:41:33 +02:00
subtle4553
3055e69c23
[ManyVidsBridge] Fix parsing of URL input (#4499) 2025-03-27 21:02:12 +01:00
tillcash
7c1e01b45a
[MinecraftBridge] Add Bridge (#4497) 2025-03-26 19:46:02 +01:00
Dag
4d8a46d46e
feat: add sanity check for required curl module (#4495) 2025-03-26 00:07:33 +01:00
Dag
9d6aa5ee38
fix: operator precedence bug (#4494) 2025-03-25 23:52:47 +01:00
subtle4553
1c45eff505
[ManyVidsBridge] Create proper feed content (#4493) 2025-03-25 23:34:19 +01:00
Joseph
68ff39e164
[TheFarSideBridge] Remove hotlink protection bypass (#4492) 2025-03-25 21:55:09 +01:00
mruac
abb1602524
fix #4475 (#4491)
* support embeds for feeds, lists and starter packs

* lint
2025-03-25 21:54:25 +01:00
Pavel Korytov
87112497de
[AnthropicBridge] Delete bridges (#4490) 2025-03-25 21:52:53 +01:00
Niehztog
38bb5115c9
fix issues reported in https://github.com/RSS-Bridge/rss-bridge/issues/4477 (#4488) 2025-03-24 21:12:26 +01:00
Tomasz Molski
23cb9349fc
[CeskaTelevizeBridge] Adjusted getting article timestamp (#4486)
* [CeskaTelevizeBridge] Adjusted getting article timestamp

* [CeskaTelevizeBridge] Removed excess whitespace
2025-03-23 21:30:45 +01:00
Pavel Korytov
05a9ac0f06
[OpenCVEBridge] Rewrite for API change (#4476)
* [OpenCVEBridge] Rewrite for API change

* [OpenCVEBridge] Fix lint
2025-03-23 21:01:21 +01:00
Dan Wainwright
91fe6c1fae
[BazarakiBridge] Add new bridge (#4473)
* [BazarakiBridge] Add new bridge

* fix

---------

Co-authored-by: Dag <me@dvikan.no>
2025-03-23 20:57:17 +01:00
chibicitiberiu
7260f28e10
[RedditBridge] Added time interval and filter for min comment count (#4471)
* Reddit Bridge - added filter for min comment count and time interval.

* [RedditBridge] Add sort by comment count

* lint

* consistent commas

---------

Co-authored-by: Dag <me@dvikan.no>
2025-03-23 20:45:35 +01:00
Tomasz Molski
87ab1e4513
[BruegelBridge] Initial commit (#4470) 2025-03-23 19:50:11 +01:00
André Andersson
dee734d360
Add Auctionet bridge (#4452) 2025-03-05 19:41:24 +01:00
Latz
744f996224
Added bridge for Toms Touché (https://taz.de/#!tom=tomdestages) (#4438) 2025-03-05 19:39:18 +01:00
Pavel Korytov
f270cd35e7
[TldrTechBridge] Fix duplicate entries and empty sections (#4466) 2025-03-05 19:36:41 +01:00
Tomasz Molski
83c36a87e2
[ReutersBridge] Adjust Fact Check feed path (#4465) 2025-03-05 19:35:12 +01:00
Tomasz Molski
810e17b556
feat: added LeagueOfLegendsNewsBridge (#4462) 2025-03-05 19:34:35 +01:00
sysadminstory
97f07cf216
[InstagramBridge] Add a fallback to the "Username" mode (#4461)
- Added some header that could help Instagram to not block RSS Bridge
- Added a fallback function to use the "Embed profile" Instagram feature
  to get the content shared by one Instagram user
2025-03-05 19:32:03 +01:00
sysadminstory
62fafdc24b
[FreeTelechargerBridge] Update URL and some fix (#4459)
- Updated the URL to the new URL in the bridge Meta Data
- Use an other URL that seems to permit to bypass CF protection
  (sometimes)
2025-03-05 19:30:38 +01:00
sysadminstory
cd4cdcfd65
[RadioMelodieBridge] Fix media content (#4458)
- Fix the audio source with the absolute URL
- Fix the pictture enclosure URL (those are already absolute URL)
2025-03-05 19:30:09 +01:00
Tobias Alexander Franke
00a24e2f69
New bridge for the latest Shadertoy submissions (#4456)
Some checks failed
Build Image on Commit and Release / bake (push) Has been cancelled
Lint / phpcs (7.4) (push) Has been cancelled
Lint / phpcompatibility (7.4) (push) Has been cancelled
Lint / executable_php_files_check (push) Has been cancelled
Tests / phpunit8 (7.4) (push) Has been cancelled
Tests / phpunit8 (8.0) (push) Has been cancelled
Tests / phpunit8 (8.1) (push) Has been cancelled
* New bridge for the latest Shadertoy submissions

* [ShadertoyBridge] Linter fixes

* [ShadertoyBridge] More Linter fixes

* [ShadertoyBridge] Even more Linter fixes
2025-02-26 10:20:28 +01:00
André Andersson
92b5e7093f
Fix data-lot-id not being correctly set so use href instead (#4453) 2025-02-24 17:58:24 +01:00
Dag
b52f01505d
fix(github): semi-repair (#4449) 2025-02-14 02:42:23 +01:00
Dag
e4c32bb046
fix(vk): semi-disable broken bridge (#4448) 2025-02-14 02:00:07 +01:00
Christian Schabesberger
dd4dcfa59c
fix nn.de description and paywall filter (#4444) 2025-02-08 01:41:51 +01:00
Tostiman
4e678c955f
fix CarThrottleBridge (#4442) 2025-02-05 18:41:42 +01:00
July
549bed64d2
[YouTubeFeedExpanderBridge] Add bridge (#4430) 2025-02-04 20:11:43 +01:00
sysadminstory
94924d8e16
[PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Fix parameters typo (#4439)
Fixed typo in DealabsBridge and HotUKDealsBridge parameters name
2025-02-03 23:24:42 +01:00
sysadminstory
920b21b1fd
[PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Fixing bridge and add subcategories (#4436)
- Follow site change to get deal data (fix for #4432)
- Add Categories (sub categories in reality) support
2025-02-03 15:35:48 +01:00
Dag
935075072b
fix: set default cache ttl of 1d (#4434) 2025-01-30 21:05:17 +01:00
July
3ae7a10223
[GovTrackBridge] Rebase on top of official RSS feed (#4429) 2025-01-29 11:11:25 +01:00
Tone
bf431a6eae
[AnisearchBridge] changed id of div so trailers work again (#4428) 2025-01-27 21:55:34 +01:00
Dag
824ac5e373
docs (#4427)
* docs

* docs
2025-01-26 21:24:33 +01:00
Bartosz Sosna
ae8394d976
Fix lfc.pl bug with page content when comments exist (#4425)
* Add lfc.pl bridge

* Adjust bridge

* Add comments section

* Fix a bug with page content when comments exist

* Add brtsos to CONTRIBUTORS.md
2025-01-26 18:58:03 +01:00
48 changed files with 1987 additions and 511 deletions

View File

@ -49,9 +49,9 @@ Please describe what you expect from the bridge. Whenever possible provide sampl
- _Default limit_: 5
- [ ] Load full articles
- _Cache articles_ (articles are stored in a local cache on first request): yes
- _Cache timeout_ (max = 24 hours): 24 hours
- _Cache timeout_ : 24 hours
- [X] Balance requests (RSS-Bridge uses cached versions to reduce bandwith usage)
- _Timeout_ (default = 5 minutes, max = 24 hours): 5 minutes
- _Timeout_ (default = 5 minutes): 5 minutes
<!--Be aware that some options might not be available for your specific request due to technical limitations!-->

View File

@ -23,6 +23,7 @@
* [Binnette](https://github.com/Binnette)
* [BoboTiG](https://github.com/BoboTiG)
* [Bockiii](https://github.com/Bockiii)
* [brtsos](https://github.com/brtsos)
* [captn3m0](https://github.com/captn3m0)
* [chemel](https://github.com/chemel)
* [Chouchen](https://github.com/Chouchen)

View File

@ -150,11 +150,11 @@ listen = /run/php/rss-bridge.sock
listen.owner = www-data
listen.group = www-data
# Create 10 workers standing by to serve requests
; Create 10 workers standing by to serve requests
pm = static
pm.max_children = 10
# Respawn worker after 500 requests (workaround for memory leaks etc.)
; Respawn worker after 500 requests (workaround for memory leaks etc.)
pm.max_requests = 500
```
@ -460,7 +460,6 @@ See [CONTRIBUTORS.md](CONTRIBUTORS.md)
RSS-Bridge uses caching to prevent services from banning your server for repeatedly updating feeds.
The specific cache duration can be different between bridges.
Cached files are deleted automatically after 24 hours.
RSS-Bridge allows you to take full control over which bridges are displayed to the user.
That way you can host your own RSS-Bridge service with your favorite collection of bridges!

View File

@ -27,6 +27,13 @@ class AO3Bridge extends BridgeAbstract
'Entire work' => 'all',
],
],
'unique' => [
'name' => 'Make separate entries for new fic chapters',
'type' => 'checkbox',
'required' => false,
'title' => 'Make separate entries for new fic chapters',
'defaultValue' => 'checked',
],
'limit' => self::LIMIT,
],
'Bookmarks' => [
@ -118,7 +125,12 @@ class AO3Bridge extends BridgeAbstract
$chapters = $element->find('dl dd.chapters', 0);
// bookmarked series and external works do not have a chapters count
$chapters = (isset($chapters) ? $chapters->plaintext : 0);
$item['uid'] = $item['uri'] . "/$strdate/$chapters";
if ($this->getInput('unique')) {
$item['uid'] = $item['uri'] . "/$strdate/$chapters";
} else {
$item['uid'] = $item['uri'];
}
// Fetch workskin of desired chapter(s) in list
if ($this->getInput('range') && ($limit == 0 || $count++ < $limit)) {

View File

@ -67,7 +67,7 @@ class AnisearchBridge extends BridgeAbstract
$trailerlink = $domarticle->find('section#trailers > div > div.swiper > ul.swiper-wrapper > li.swiper-slide > a', 0);
if (isset($trailerlink)) {
$trailersite = getSimpleHTMLDOM($baseurl . $trailerlink->href);
$trailer = $trailersite->find('div#player > iframe', 0);
$trailer = $trailersite->find('div#video > iframe', 0);
$trailer = $trailer->{'data-xsrc'};
$ytlink = <<<EOT
<br /><iframe width="560" height="315" src="$trailer" title="YouTube video player"

View File

@ -1,147 +0,0 @@
<?php
class AnthropicBridge extends BridgeAbstract
{
const MAINTAINER = 'sqrtminusone';
const NAME = 'Anthropic Research Bridge';
const URI = 'https://www.anthropic.com';
const CACHE_TIMEOUT = 3600; // 1 hour
const DESCRIPTION = 'Returns research publications from Anthropic';
const PARAMETERS = [
'' => [
'limit' => [
'name' => 'Limit',
'type' => 'number',
'required' => true,
'defaultValue' => 10
],
]
];
public function collectData()
{
// Anthropic sometimes returns 500 for no reason. The contents are still there.
$html = $this->getHTMLIgnoreError(self::URI . '/research');
$limit = $this->getInput('limit');
$page_data = $this->extractPageData($html);
$pages = $this->parsePageData($page_data);
for ($i = 0; $i < min(count($pages), $limit); $i++) {
$page = $pages[$i];
$page['content'] = $this->parsePage($page['uri']);
$this->items[] = $page;
}
}
private function getHTMLIgnoreError($url, $ttl = null)
{
if ($ttl != null) {
$cacheKey = 'pages_' . $url;
$content = $this->cache->get($cacheKey);
if ($content) {
return str_get_html($content);
}
}
try {
$content = getContents($url);
} catch (HttpException $e) {
$content = $e->response->getBody();
}
if ($ttl != null) {
$this->cache->set($cacheKey, $content, $ttl);
}
return str_get_html($content);
}
private function extractPageData($html)
{
foreach ($html->find('script') as $script) {
$js_code = $script->innertext;
if (!str_starts_with($js_code, 'self.__next_f.push(')) {
continue;
}
$push_data = (string)json_decode(mb_substr($js_code, 22, mb_strlen($js_code) - 2 - 22));
$square_bracket = mb_strpos($push_data, '[');
$push_array = json_decode(mb_substr($push_data, $square_bracket), true);
if ($push_array == null || count($push_array) < 4) {
continue;
}
$page_data = $push_array[3];
if ($page_data != null && array_key_exists('page', $page_data)) {
return $page_data;
}
}
}
private function parsePageData($page_data)
{
$result = [];
foreach ($page_data['page']['sections'] as $section) {
if (
!array_key_exists('internalName', $section) ||
$section['internalName'] != 'Research Teams'
) {
continue;
}
foreach ($section['tabPages'] as $tabPage) {
if ($tabPage['label'] != 'Overview') {
continue;
}
foreach ($tabPage['sections'] as $section1) {
if (
!array_key_exists('title', $section1)
|| $section1['title'] != 'Publications'
) {
continue;
}
foreach ($section1['posts'] as $post) {
$enc = [];
if ($post['cta'] != null && array_key_exists('url', $post['cta'])) {
$enc = [$post['cta']['url']];
}
$result[] = [
'title' => $post['title'],
'timestamp' => $post['publishedOn'],
'uri' => self::URI . '/research/' . $post['slug']['current'],
'categories' => array_map(
fn($s) => $s['label'],
$post['subjects'],
),
'enclosures' => $enc,
];
}
break;
}
break;
}
break;
}
return $result;
}
private function parsePage($url)
{
// Again, 500 for no reason.
$html = $this->getHTMLIgnoreError($url, 7 * 24 * 60 * 60);
$content = '';
// Main content
$main = $html->find('div[class*="PostDetail_post-detail"] > article', 0);
// Mostly YouTube videos
$iframes = $main->find('iframe');
foreach ($iframes as $iframe) {
$iframe->parent->removeAttribute('style');
$iframe->outertext = '<a href="' . $iframe->src . '">' . $iframe->src . '</a>';
}
$main = convertLazyLoading($main);
$main = defaultLinkTo($main, self::URI);
$content .= $main;
return $content;
}
}

344
bridges/AuctionetBridge.php Normal file
View File

@ -0,0 +1,344 @@
<?php
class AuctionetBridge extends BridgeAbstract
{
const NAME = 'Auctionet';
const URI = 'https://www.auctionet.com';
const DESCRIPTION = 'Fetches info about auction objects from Auctionet (an auction platform for many European auction houses)';
const MAINTAINER = 'Qluxzz';
const PARAMETERS = [[
'category' => [
'name' => 'Category',
'type' => 'list',
'values' => [
'All categories' => '',
'Art' => [
'All' => '25-art',
'Drawings' => '119-drawings',
'Engravings & Prints' => '27-engravings-prints',
'Other' => '30-other',
'Paintings' => '28-paintings',
'Photography' => '26-photography',
'Sculptures & Bronzes' => '29-sculptures-bronzes',
],
'Asiatica' => [
'All' => '117-asiatica',
],
'Books, Maps & Manuscripts' => [
'All' => '50-books-maps-manuscripts',
'Autographs & Manuscripts' => '206-autographs-manuscripts',
'Books' => '204-books',
'Maps' => '205-maps',
'Other' => '207-other',
],
'Carpets & Textiles' => [
'All' => '35-carpets-textiles',
'Carpets' => '36-carpets',
'Textiles' => '37-textiles',
],
'Ceramics & Porcelain' => [
'All' => '9-ceramics-porcelain',
'European' => '10-european',
'Oriental' => '11-oriental',
'Rest of the world' => '12-rest-of-the-world',
'Tableware' => '210-tableware',
],
'Clocks & Watches' => [
'All' => '31-clocks-watches',
'Carriage & Miniature Clocks' => '258-carriage-miniature-clocks',
'Longcase clocks' => '32-longcase-clocks',
'Mantel clocks' => '33-mantel-clocks',
'Other clocks' => '34-other-clocks',
'Pocket & Stop Watches' => '110-pocket-stop-watches',
'Wall Clocks' => '127-wall-clocks',
'Wristwatches' => '15-wristwatches',
],
'Coins, Medals & Stamps' => [
'All' => '46-coins-medals-stamps',
'Coins' => '128-coins',
'Orders & Medals' => '135-orders-medals',
'Other' => '131-other',
'Stamps' => '136-stamps',
],
'Folk art' => [
'All' => '58-folk-art',
'Bowls & Boxes' => '121-bowls-boxes',
'Furniture' => '122-furniture',
'Other' => '123-other',
'Tools & Gears' => '120-tools-gears',
],
'Furniture' => [
'All' => '16-furniture',
'Armchairs & Chairs' => '18-armchairs-chairs',
'Chests of drawers' => '24-chests-of-drawers',
'Cupboards, Cabinets & Shelves' => '23-cupboards-cabinets-shelves',
'Dining room furniture' => '22-dining-room-furniture',
'Garden' => '21-garden',
'Other' => '17-other',
'Sofas & seatings' => '20-sofas-seatings',
'Tables' => '19-tables',
],
'Glass' => [
'All' => '6-glass',
'Art glass' => '208-art-glass',
'Other' => '8-other',
'Tableware' => '7-tableware',
'Utility glass' => '209-utility-glass',
],
'Jewellery & Gemstones' => [
'All' => '13-jewellery-gemstones',
'Alliance rings' => '113-alliance-rings',
'Bracelets' => '106-bracelets',
'Brooches & Pendants' => '107-brooches-pendants',
'Costume Jewellery' => '259-costume-jewellery',
'Cufflinks & Tie Pins' => '111-cufflinks-tie-pins',
'Ear studs' => '116-ear-studs',
'Earrings' => '115-earrings',
'Gemstones' => '48-gemstones',
'Jewellery' => '14-jewellery',
'Jewellery Suites' => '109-jewellery-suites',
'Necklace' => '104-necklace',
'Other' => '118-other',
'Rings' => '112-rings',
'Signet rings' => '105-signet-rings',
'Solitaire rings' => '114-solitaire-rings',
],
'Licence weapons' => [
'All' => '59-licence-weapons',
'Combi/Combo' => '63-combi-combo',
'Double express rifles' => '60-double-express-rifles',
'Rifles' => '61-rifles',
'Shotguns' => '62-shotguns',
],
'Lighting & Lamps' => [
'All' => '1-lighting-lamps',
'Candlesticks' => '4-candlesticks',
'Ceiling lights' => '3-ceiling-lights',
'Chandeliers' => '203-chandeliers',
'Floor lights' => '2-floor-lights',
'Other lighting' => '5-other-lighting',
'Table Lamps' => '125-table-lamps',
'Wall Lights' => '124-wall-lights',
],
'Mirrors' => [
'All' => '42-mirrors',
],
'Miscellaneous' => [
'All' => '43-miscellaneous',
'Fishing equipment' => '54-fishing-equipment',
'Miscellaneous' => '47-miscellaneous',
'Modern Tools' => '133-modern-tools',
'Modern consumer electronics' => '52-modern-consumer-electronics',
'Musical instruments' => '51-musical-instruments',
'Technica & Nautica' => '45-technica-nautica',
],
'Photo, Cameras & Lenses' => [
'All' => '57-photo-cameras-lenses',
'Cameras & accessories' => '71-cameras-accessories',
'Optics' => '66-optics',
'Other' => '72-other',
],
'Silver & Metals' => [
'All' => '38-silver-metals',
'Other metals' => '40-other-metals',
'Pewter, Brass & Copper' => '41-pewter-brass-copper',
'Silver' => '39-silver',
'Silver plated' => '213-silver-plated',
],
'Toys' => [
'All' => '44-toys',
'Comics' => '211-comics',
'Toys' => '212-toys',
],
'Tribal art' => [
'All' => '134-tribal-art',
],
'Vehicles, Boats & Parts' => [
'All' => '249-vehicles-boats-parts',
'Automobilia & Transport' => '255-automobilia-transport',
'Bicycles' => '132-bicycles',
'Boats & Accessories' => '250-boats-accessories',
'Car parts' => '253-car-parts',
'Cars' => '215-cars',
'Moped parts' => '254-moped-parts',
'Mopeds' => '216-mopeds',
'Motorcycle parts' => '252-motorcycle-parts',
'Motorcycles' => '251-motorcycles',
'Other' => '256-other',
],
'Vintage & Designer Fashion' => [
'All' => '49-vintage-designer-fashion',
],
'Weapons & Militaria' => [
'All' => '137-weapons-militaria',
'Airguns' => '257-airguns',
'Armour & Uniform' => '138-armour-uniform',
'Edged weapons' => '130-edged-weapons',
'Guns & Rifles' => '129-guns-rifles',
'Other' => '214-other',
],
'Wine, Port & Spirits' => [
'All' => '170-wine-port-spirits',
],
]
],
'sort_order' => [
'name' => 'Sort order',
'type' => 'list',
'values' => [
'Most bids' => 'bids_count_desc',
'Lowest bid' => 'bid_asc',
'Highest bid' => 'bid_desc',
'Last bid on' => 'bid_on',
'Ending soonest' => 'end_asc_active',
'Lowest estimate' => 'estimate_asc',
'Highest estimate' => 'estimate_desc',
'Recently added' => 'recent'
],
],
'country' => [
'name' => 'Country',
'type' => 'list',
'values' => [
'All' => '',
'Denmark' => 'DK',
'Finland' => 'FI',
'Germany' => 'DE',
'Spain' => 'ES',
'Sweden' => 'SE',
'United Kingdom' => 'GB'
]
],
'language' => [
'name' => 'Language',
'type' => 'list',
'values' => [
'English' => 'en',
'Español' => 'es',
'Deutsch' => 'de',
'Svenska' => 'sv',
'Dansk' => 'da',
'Suomi' => 'fi',
],
],
]];
const CACHE_TIMEOUT = 3600; // 1 hour
private $title;
public function collectData()
{
// Each page contains 48 auctions
// So we fetch 10 pages so we decrease the likelihood
// of missing auctions between feed refreshes
// Fetch first page and use that to get title
{
$url = $this->getUrl(1);
$data = getContents($url);
$title = $this->getDocumentTitle($data);
$this->items = array_merge($this->items, $this->parsePageData($data));
}
// Fetch remaining pages
for ($page = 2; $page <= 10; $page++) {
$url = $this->getUrl($page);
$data = getContents($url);
$this->items = array_merge($this->items, $this->parsePageData($data));
}
}
public function getName()
{
return $this->title ?: parent::getName();
}
/* HELPERS */
private function getUrl($page)
{
$category = $this->getInput('category');
$language = $this->getInput('language');
$sort_order = $this->getInput('sort_order');
$country = $this->getInput('country');
$url = self::URI . '/' . $language . '/search';
if ($category) {
$url = $url . '/' . $category;
}
$query = [];
$query['page'] = $page;
if ($sort_order) {
$query['order'] = $sort_order;
}
if ($country) {
$query['country_code'] = $country;
}
if (count($query) > 0) {
$url = $url . '?' . http_build_query($query);
}
return $url;
}
private function getDocumentTitle($data)
{
$title_elem = '<title>';
$title_elem_length = strlen($title_elem);
$title_start = strpos($data, $title_elem);
$title_end = strpos($data, '</title>', $title_start);
$title_length = $title_end - $title_start + strlen($title_elem);
$title = substr($data, $title_start + strlen($title_elem), $title_length);
return $title;
}
/**
* The auction items data is included in the HTML document
* as a HTML entities encoded JSON structure
* which is used to hydrate the React component for the list of auctions
*/
private function parsePageData($data)
{
$key = 'data-react-props="';
$keyLength = strlen($key);
$start = strpos($data, $key);
$end = strpos($data, '"', $start + strlen($key));
$length = $end - ($start + $keyLength);
$jsonString = substr($data, $start + $keyLength, $length);
$jsonData = json_decode(htmlspecialchars_decode($jsonString), false);
$items = [];
foreach ($jsonData->{'items'} as $item) {
$title = $item->{'longTitle'};
$relative_url = $item->{'url'};
$images = $item->{'imageUrls'};
$id = $item->{'auctionId'};
$items[] = [
'title' => $title,
'uri' => self::URI . $relative_url,
'uid' => $id,
'content' => count($images) > 0 ? "<img src='$images[0]'/><br/>$title" : $title,
'enclosures' => array_slice($images, 1),
];
}
return $items;
}
}

139
bridges/BazarakiBridge.php Normal file
View File

@ -0,0 +1,139 @@
<?php
class BazarakiBridge extends BridgeAbstract
{
const NAME = 'Bazaraki Bridge';
const URI = 'https://bazaraki.com';
const DESCRIPTION = 'Fetch adverts from Bazaraki, a Cyprus-based classifieds website.';
const MAINTAINER = 'danwain';
const PARAMETERS = [
[
'url' => [
'name' => 'URL',
'type' => 'text',
'required' => true,
'title' => 'Enter the URL of the Bazaraki page to fetch adverts from.',
'exampleValue' => 'https://www.bazaraki.com/real-estate-for-sale/houses/?lat=0&lng=0&radius=100000',
],
'limit' => [
'name' => 'Limit',
'type' => 'number',
'required' => false,
'title' => 'Enter the number of adverts to fetch. (max 50)',
'exampleValue' => '10',
'defaultValue' => 10,
]
]
];
public function collectData()
{
$url = $this->getInput('url');
if (! str_starts_with($url, 'https://www.bazaraki.com/')) {
throw new \Exception('Nope');
}
$html = getSimpleHTMLDOM($url);
$i = 0;
foreach ($html->find('div.advert') as $element) {
$i++;
if ($i > $this->getInput('limit') || $i > 50) {
break;
}
$item = [];
$item['uri'] = 'https://www.bazaraki.com' . $element->find('a.advert__content-title', 0)->href;
# Get the content
$advert = getSimpleHTMLDOM($item['uri']);
$price = trim($advert->find('div.announcement-price__cost', 0)->plaintext);
$name = trim($element->find('a.advert__content-title', 0)->plaintext);
$item['title'] = $name . ' - ' . $price;
$time = trim($advert->find('span.date-meta', 0)->plaintext);
$time = str_replace('Posted: ', '', $time);
$item['content'] = $this->processAdvertContent($advert);
$item['timestamp'] = $this->convertRelativeTime($time);
$item['author'] = trim($advert->find('div.author-name', 0)->plaintext);
$item['uid'] = $advert->find('span.number-announcement', 0)->plaintext;
$this->items[] = $item;
}
}
/**
* Process the advert content to clean up HTML
*
* @param simple_html_dom $advert The SimpleHTMLDOM object for the advert page
* @return string Processed HTML content
*/
private function processAdvertContent($advert)
{
// Get the content sections
$header = $advert->find('div.announcement-content-header', 0);
$characteristics = $advert->find('div.announcement-characteristics', 0);
$description = $advert->find('div.js-description', 0);
$images = $advert->find('div.announcement__images', 0);
// Remove all favorites divs
foreach ($advert->find('div.announcement-meta__favorites') as $favorites) {
$favorites->outertext = '';
}
// Replace all <a> tags with their text content
foreach ($advert->find('a') as $a) {
$a->outertext = $a->innertext;
}
// Format the content with section headers and dividers
$formattedContent = '';
// Add header section
$formattedContent .= $header->innertext;
$formattedContent .= '<hr/>';
// Add characteristics section with header
$formattedContent .= '<h3>Details</h3>';
$formattedContent .= $characteristics->innertext;
$formattedContent .= '<hr/>';
// Add description section with header
$formattedContent .= '<h3>Description</h3>';
$formattedContent .= $description->innertext;
$formattedContent .= '<hr/>';
// Add images section with header
$formattedContent .= '<h3>Images</h3>';
$formattedContent .= $images->innertext;
return $formattedContent;
}
/**
* Convert relative time strings like "Yesterday 12:32" to proper timestamps
*
* @param string $timeString The relative time string from the website
* @return string Timestamp in a format compatible with strtotime()
*/
private function convertRelativeTime($timeString)
{
if (strpos($timeString, 'Yesterday') !== false) {
// Replace "Yesterday" with actual date
$time = str_replace('Yesterday', date('Y-m-d', strtotime('-1 day')), $timeString);
return date('Y-m-d H:i:s', strtotime($time));
} elseif (strpos($timeString, 'Today') !== false) {
// Replace "Today" with actual date
$time = str_replace('Today', date('Y-m-d'), $timeString);
return date('Y-m-d H:i:s', strtotime($time));
} else {
// For other formats, return as is and let strtotime handle it
return $timeString;
}
}
}

View File

@ -180,7 +180,7 @@ class BlueskyBridge extends BridgeAbstract
if (Debug::isEnabled()) {
$url = explode('/', $post['post']['uri']);
error_log('https://bsky.app/profile/' . $url[2] . '/post/' . $url[4]);
$this->logger->debug('https://bsky.app/profile/' . $url[2] . '/post/' . $url[4]);
}
$description = '';
@ -255,10 +255,16 @@ class BlueskyBridge extends BridgeAbstract
$description .= '<a href="' . $uri_reconstructed . '">Quoted post detached.</a>';
} elseif (isset($quotedRecord['blocked']) && $quotedRecord['blocked']) { //blocked by quote author
$description .= 'Author of quoted post has blocked OP.';
} elseif (($quotedRecord['$type'] ?? '') === 'app.bsky.feed.defs#generatorView') {
$description .= '</p>';
$description .= $this->getGeneratorViewDescription($quotedRecord);
$description .= '<p>';
} elseif (
($quotedRecord['$type'] ?? '') === 'app.bsky.feed.defs#generatorView' ||
($quotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#listView'
) {
$description .= $this->getListFeedDescription($quotedRecord);
} elseif (
($quotedRecord['$type'] ?? '') === 'app.bsky.graph.starterpack' ||
($quotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#starterPackViewBasic'
) {
$description .= $this->getStarterPackDescription($post['post']['embed']['record']);
} else {
$quotedAuthorDid = $quotedRecord['author']['did'];
$quotedDisplayName = $quotedRecord['author']['displayName'] ?? '';
@ -403,10 +409,16 @@ class BlueskyBridge extends BridgeAbstract
$description .= '<a href="' . $uri_reconstructed . '">Quoted post detached.</a>';
} elseif (isset($replyQuotedRecord['blocked']) && $replyQuotedRecord['blocked']) { //blocked by quote author
$description .= 'Author of quoted post has blocked OP.';
} elseif (($replyQuotedRecord['$type'] ?? '') === 'app.bsky.feed.defs#generatorView') {
$description .= '</p>';
$description .= $this->getGeneratorViewDescription($replyQuotedRecord);
$description .= '<p>';
} elseif (
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.feed.defs#generatorView' ||
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#listView'
) {
$description .= $this->getListFeedDescription($replyQuotedRecord);
} elseif (
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.starterpack' ||
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#starterPackViewBasic'
) {
$description .= $this->getStarterPackDescription($replyPost['embed']['record']);
} else {
$quotedAuthorDid = $replyQuotedRecord['author']['did'];
$quotedDisplayName = $replyQuotedRecord['author']['displayName'] ?? '';
@ -554,11 +566,19 @@ class BlueskyBridge extends BridgeAbstract
}
$title .= ', replying to ' . $replyAuthor;
}
if (isset($post['post']['embed']) && isset($post['post']['embed']['record'])) {
if (
isset($post['post']['embed']) &&
isset($post['post']['embed']['record']) &&
//if not starter pack, feed or list
($post['post']['embed']['record']['$type'] ?? '') !== 'app.bsky.feed.defs#generatorView' &&
($post['post']['embed']['record']['$type'] ?? '') !== 'app.bsky.graph.defs#listView' &&
($post['post']['embed']['record']['$type'] ?? '') !== 'app.bsky.graph.defs#starterPackViewBasic'
) {
if (isset($post['post']['embed']['record']['blocked'])) {
$quotedAuthor = 'blocked user';
} elseif (isset($post['post']['embed']['record']['notFound'])) {
$quotedAuthor = 'deleted post';
$quotedAuthor = 'deleted psost';
} elseif (isset($post['post']['embed']['record']['detached'])) {
$quotedAuthor = 'detached post';
} else {
@ -587,34 +607,64 @@ class BlueskyBridge extends BridgeAbstract
{
$uri = 'https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed?actor=' . urlencode($did) . '&filter=' . urlencode($filter) . '&limit=30';
if (Debug::isEnabled()) {
error_log($uri);
$this->logger->debug($uri);
}
$response = json_decode(getContents($uri), true);
return $response;
}
private function getGeneratorViewDescription(array $record): string
//Embed for generated feeds and lists
private function getListFeedDescription(array $record): string
{
$avatar = e($record['avatar']);
$displayName = e($record['displayName']);
$displayHandle = e($record['creator']['handle']);
$likeCount = e($record['likeCount']);
$feedViewAvatar = isset($record['avatar']) ? '<img src="' . preg_replace('/\/img\/avatar\//', '/img/avatar_thumbnail/', $record['avatar']) . '">' : '';
$feedViewName = e($record['displayName'] ?? $record['name']);
$feedViewDescription = e($record['description'] ?? '');
$authorDisplayName = e($record['creator']['displayName']);
$authorHandle = e($record['creator']['handle']);
$likeCount = isset($record['likeCount']) ? '<br>Liked by ' . e($record['likeCount']) . ' users' : '';
preg_match('/\/([^\/]+)$/', $record['uri'], $matches);
$uri = e('https://bsky.app/profile/' . $record['creator']['did'] . '/feed/' . $matches[1]);
if (($record['purpose'] ?? '') === 'app.bsky.graph.defs#modlist') {
$typeURL = '/lists/';
$typeDesc = 'moderation list';
} elseif (($record['purpose'] ?? '') === 'app.bsky.graph.defs#curatelist') {
$typeURL = '/lists/';
$typeDesc = 'list';
} else {
$typeURL = '/feed/';
$typeDesc = 'feed';
}
$uri = e('https://bsky.app/profile/' . $record['creator']['did'] . $typeURL . $matches[1]);
return <<<END
<a href="{$uri}" style="color: inherit;">
<div style="border: 1px solid #333; padding: 10px;">
<div style="display: flex; margin-bottom: 10px;">
<img src="{$avatar}" height="50" width="50" style="margin-right: 10px;">
<div style="display: flex; flex-direction: column; justify-content: center;">
<h3>{$displayName}</h3>
<span>Feed by @{$displayHandle}</span>
</div>
</div>
<span>Liked by {$likeCount} users</span>
</div>
</a>
<blockquote>
<b><a href="{$uri}">{$feedViewName}</a></b><br/>
Bluesky {$typeDesc} by <b>{$authorDisplayName}</b> <i>@{$authorHandle}</i>
<figure>
{$feedViewAvatar}
<figcaption>{$feedViewDescription}{$likeCount}</figcaption>
</figure>
</blockquote>
END;
}
private function getStarterPackDescription(array $record): string
{
if (!isset($record['record'])) {
return 'Failed to get starter pack information.';
}
$starterpackRecord = $record['record'];
$starterpackName = e($starterpackRecord['name']);
$starterpackDescription = e($starterpackRecord['description']);
$creatorDisplayName = e($record['creator']['displayName']);
$creatorHandle = e($record['creator']['handle']);
preg_match('/\/([^\/]+)$/', $starterpackRecord['list'], $matches);
$uri = e('https://bsky.app/starter-pack/' . $record['creator']['did'] . '/' . $matches[1]);
return <<<END
<blockquote>
<b><a href="{$uri}">{$starterpackName}</a></b><br/>
Bluesky starter pack by <b>{$creatorDisplayName}</b> <i>@{$creatorHandle}</i><br/>
{$starterpackDescription}
</blockquote>
END;
}
}

63
bridges/BruegelBridge.php Normal file
View File

@ -0,0 +1,63 @@
<?php
class BruegelBridge extends BridgeAbstract
{
const NAME = 'Bruegel';
const URI = 'https://www.bruegel.org';
const DESCRIPTION = 'European think-tank commentary and publications.';
const MAINTAINER = 'KappaPrajd';
const PARAMETERS = [
[
'category' => [
'name' => 'Category',
'type' => 'list',
'defaultValue' => '/publications',
'values' => [
'Publications' => '/publications',
'Commentary' => '/commentary'
]
]
]
];
public function getIcon()
{
return self::URI . '/themes/custom/bruegel/assets/favicon/android-icon-72x72.png';
}
public function collectData()
{
$url = self::URI . $this->getInput('category');
$html = getSimpleHTMLDOM($url);
$articles = $html->find('.c-listing__content article');
foreach ($articles as $article) {
$title = $article->find('.c-list-item__title a span', 0)->plaintext;
$content = trim($article->find('.c-list-item__description', 0)->plaintext);
$publishDate = $article->find('.c-list-item__date', 0)->plaintext;
$href = $article->find('.c-list-item__title a', 0)->getAttribute('href');
$item = [
'title' => $title,
'content' => $content,
'timestamp' => strtotime($publishDate),
'uri' => self::URI . $href,
'author' => $this->getAuthor($article),
];
$this->items[] = $item;
}
}
private function getAuthor($article)
{
$authorsElements = $article->find('.c-list-item__authors a');
$authors = array_map(function ($author) {
return $author->plaintext;
}, $authorsElements);
return join(', ', $authors);
}
}

View File

@ -206,7 +206,7 @@ class BukowskisBridge extends BridgeAbstract
$this->items[] = [
'title' => $title,
'uri' => $baseUrl . $relative_url,
'uid' => $lot->getAttribute('data-lot-id'),
'uid' => $relative_url,
'content' => count($images) > 0 ? "<img src='$images[0]'/><br/>$title" : $title,
'enclosures' => array_slice($images, 1),
];

View File

@ -66,7 +66,7 @@ class CarThrottleBridge extends BridgeAbstract
foreach ($categoryPage->find('div.cmg-card') as $post) {
$item = [];
$titleElement = $post->find('div.title a')[0];
$titleElement = $post->find('a.title')[0];
$post_uri = self::URI . $titleElement->getAttribute('href');
if (!isset($post_uri) || $post_uri == '') {
@ -80,8 +80,8 @@ class CarThrottleBridge extends BridgeAbstract
$item['author'] = $this->parseAuthor($articlePage);
$articleImage = $articlePage->find('div.block-layout-field-image')[0];
$article = $articlePage->find('div.block-layout-body')[1];
$articleImage = $articlePage->find('figure')[0];
$article = $articlePage->find('div.first-column div.body')[0];
//remove ads
foreach ($article->find('aside') as $ad) {

View File

@ -41,11 +41,15 @@ class CeskaTelevizeBridge extends BridgeAbstract
foreach ($html->find('#episodeListSection a[data-testid=card]') as $element) {
$itemContent = $element->find('p[class^=content-]', 0);
$itemDate = $element->find('div[class^=playTime-] span, [data-testid=episode-item-broadcast] span', 0);
// Remove special characters and whitespace
$cleanDate = preg_replace('/[^0-9.]/', '', $itemDate->plaintext);
$item = [
'title' => $this->fixChars($element->find('h3', 0)->plaintext),
'uri' => self::URI . $element->getAttribute('href'),
'content' => '<img src="' . $element->find('img', 0)->getAttribute('srcset') . '" /><br />' . $this->fixChars($itemContent->plaintext),
'timestamp' => $this->getUploadTimeFromString($itemDate->plaintext),
'timestamp' => $this->getUploadTimeFromString($cleanDate),
];
$this->items[] = $item;
@ -58,7 +62,7 @@ class CeskaTelevizeBridge extends BridgeAbstract
return strtotime('today');
} elseif (strpos($string, 'včera') !== false) {
return strtotime('yesterday');
} elseif (!preg_match('/(\d+).\s(\d+).(\s(\d+))?/', $string, $match)) {
} elseif (!preg_match('/(\d+).(\d+).((\d+))?/', $string, $match)) {
returnServerError('Could not get date from Česká televize string');
}

View File

@ -48,6 +48,16 @@ https://www.dealabs.com/groupe/abonnements-internet?sortBy=lowest_price
Il faut alors saisir :
abonnements-internet',
],
'subgroups' => [
'name' => 'Catégorie',
'type' => 'text',
'exampleValue' => '1071',
'title' => 'Numéro du ou des catégories dans l\'URL : Il faut entrer le ou les numéros de catégories qui sont présent après "groups=" et avant tout éventuel "&"
Exemple : Si l\'URL du groupe affichées dans le navigateur est :
https://www.dealabs.com/groupe/telecommunications?groups=1071%2C1070&sortBy=new
Il faut alors saisir :
1071%2C1070',
],
'order' => [
'name' => 'Trier par',
'type' => 'list',
@ -88,6 +98,7 @@ abonnements-internet',
'uri-group' => 'groupe/',
'uri-deal' => 'bons-plans/',
'uri-merchant' => 'search/bons-plans?merchant-id=',
'image-host' => 'https://static-pepper.dealabs.com/',
'request-error' => 'Impossible de joindre Dealabs',
'thread-error' => 'Impossible de déterminer l\'ID de la discussion. Vérifiez l\'URL que vous avez entré',
'currency' => '€',

View File

@ -3,7 +3,8 @@
class FreeTelechargerBridge extends BridgeAbstract
{
const NAME = 'Free-Telecharger';
const URI = 'https://www.free-telecharger.art/';
const URI = 'https://www.free-telecharger.fun/';
const ALTERNATEURI = 'https://www.free-telecharger.com/';
const DESCRIPTION = 'Suivi de série sur Free-Telecharger';
const MAINTAINER = 'sysadminstory';
const PARAMETERS = [
@ -12,19 +13,19 @@ class FreeTelechargerBridge extends BridgeAbstract
'name' => 'URL de la série',
'type' => 'text',
'required' => true,
'title' => 'URL d\'une série sans le https://www.free-telecharger.art/',
'title' => 'URL d\'une série sans le https://www.free-telecharger.fun/',
'pattern' => 'series.*\.html',
'exampleValue' => 'series-vf-hd/151432-wolf-saison-1-complete-web-dl-720p.html'
],
]
];
const CACHE_TIMEOUT = 3600;
private string $showTitle;
private string $showTechDetails;
private string $showTitle = '';
private string $showTechDetails = '';
public function collectData()
{
$html = getSimpleHTMLDOM(self::URI . $this->getInput('url'));
$html = getSimpleHTMLDOM(self::ALTERNATEURI . $this->getInput('url'));
// Find all block content of the page
$blocks = $html->find('div[class=block1]');

View File

@ -192,15 +192,18 @@ class GithubIssueBridge extends BridgeAbstract
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI());
$url = $this->getURI();
$html = getSimpleHTMLDOM($url);
switch ($this->queriedContext) {
case static::BRIDGE_OPTIONS[1]: // Issue comments
$this->items = $this->extractIssueComments($html);
break;
case static::BRIDGE_OPTIONS[0]: // Project Issues
foreach ($html->find('.js-active-navigation-container .js-navigation-item') as $issue) {
$info = $issue->find('.opened-by', 0);
$issues = $html->find('.js-active-navigation-container .js-navigation-item');
$issues = $html->find('.IssueRow-module__row--XmR1f');
foreach ($issues as $issue) {
$info = $issue->find('.issue-item-module__authorCreatedLink--wFZvk', 0);
preg_match('/\/([0-9]+)$/', $issue->find('a', 0)->href, $match);
$issueNbr = $match[1];
@ -222,24 +225,24 @@ class GithubIssueBridge extends BridgeAbstract
$item['content'] = 'Can not extract comments from ' . $uri;
}
$item['author'] = $info->find('a', 0)->plaintext;
$item['author'] = $issue->find('a', 1)->plaintext;
$item['timestamp'] = strtotime(
$info->find('relative-time', 0)->getAttribute('datetime')
$issue->find('relative-time', 0)->getAttribute('datetime')
);
$item['title'] = html_entity_decode(
$issue->find('.js-navigation-open', 0)->plaintext,
$issue->find('h3', 0)->plaintext,
ENT_QUOTES,
'UTF-8'
);
$comment_count = 0;
if ($span = $issue->find('a[aria-label*="comment"] span', 0)) {
$comment_count = $span->plaintext;
}
//$comment_count = 0;
//if ($span = $issue->find('a[aria-label*="comment"] span', 0)) {
// $comment_count = $span->plaintext;
//}
$item['content'] .= "\n" . 'Comments: ' . $comment_count;
//$item['content'] .= "\n" . 'Comments: ' . $comment_count;
$item['uri'] = self::URI
. trim($issue->find('.js-navigation-open', 0)->getAttribute('href'), '/');
. trim($issue->find('a', 0)->getAttribute('href'), '/');
$this->items[] = $item;
}
break;

View File

@ -2,7 +2,8 @@
class GoComicsBridge extends BridgeAbstract
{
const MAINTAINER = 'sky';
const MAINTAINER = 'TReKiE';
//const MAINTAINER = 'sky';
const NAME = 'GoComics Unofficial RSS';
const URI = 'https://www.gocomics.com/';
const CACHE_TIMEOUT = 21600; // 6h
@ -13,32 +14,53 @@ class GoComicsBridge extends BridgeAbstract
'type' => 'text',
'exampleValue' => 'heartofthecity',
'required' => true
],
'date-in-title' => [
'name' => 'Add date and full name to each day\'s title',
'type' => 'checkbox',
'title' => 'Adds the date and the full name into the title of each day\'s comic',
],
'limit' => [
'name' => 'Limit',
'type' => 'number',
'title' => 'The number of recent comics to get',
'defaultValue' => 5
]
]];
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI());
$link = $this->getURI();
//Get info from first page
$author = preg_replace('/By /', '', $html->find('.media-subheading', 0)->plaintext);
for ($i = 0; $i < $this->getInput('limit'); $i++) {
$html = getSimpleHTMLDOM($link);
// get json data from the first page
$json = $html->find('div.ComicViewer_comicViewer__comic__oftX6 script[type="application/ld+json"]', 0)->innertext;
$data = json_decode($json, false);
$link = self::URI . $html->find('.gc-deck--cta-0', 0)->find('a', 0)->href;
for ($i = 0; $i < 5; $i++) {
$item = [];
$page = getSimpleHTMLDOM($link);
$imagelink = $page->find('.comic.container', 0)->getAttribute('data-image');
$date = explode('/', $link);
$author = $data->author->name;
$imagelink = $data->contentUrl;
$date = $data->datePublished;
$title = $data->name . ' - GoComics';
// get a permlink for this day's comic if there isn't one specified
if ($link === $this->getURI()) {
$link = $this->getURI() . '/' . DateTime::createFromFormat('F j, Y', $date)->format('Y/m/d');
}
$item['id'] = $imagelink;
$item['uri'] = $link;
$item['author'] = $author;
$item['title'] = 'GoComics ' . $this->getInput('comicname');
$item['timestamp'] = DateTime::createFromFormat('Ymd', $date[5] . $date[6] . $date[7])->getTimestamp();
if ($this->getInput('date-in-title') === true) {
$item['title'] = $title;
}
$item['timestamp'] = DateTime::createFromFormat('F j, Y', $date)->setTime(0, 0, 0)->getTimestamp();
$item['content'] = '<img src="' . $imagelink . '" />';
$link = self::URI . $page->find('.js-previous-comic', 0)->href;
$link = rtrim(self::URI, '/') . $html->find('.Controls_controls__button_previous__P4LhX', 0)->href;
$this->items[] = $item;
}
}

View File

@ -1,6 +1,6 @@
<?php
class GovTrackBridge extends BridgeAbstract
class GovTrackBridge extends FeedExpander
{
const NAME = 'GovTrack';
const MAINTAINER = 'phantop';
@ -18,64 +18,50 @@ class GovTrackBridge extends BridgeAbstract
'Major Legislative Activity' => 'major-bill-activity',
'New Bills and Resolutions' => 'introduced-bills',
'New Laws' => 'enacted-bills',
'Posts from Us' => 'posts'
]
],
'limit' => self::LIMIT
'News from Us' => 'posts'
]
],
'limit' => self::LIMIT
]];
public function collectData()
{
$html = getSimpleHTMLDOMCached($this->getURI());
if ($this->getInput('feed') != 'posts') {
$this->collectEvent($html);
return;
}
$html = defaultLinkTo($html, parent::getURI());
$limit = $this->getInput('limit') ?? 10;
foreach ($html->find('section') as $element) {
if (--$limit == 0) {
break;
}
$info = explode(' ', $element->find('p', 0)->innertext);
$item = [
'categories' => [implode(' ', array_slice($info, 4))],
'timestamp' => strtotime(implode(' ', array_slice($info, 0, 3))),
'title' => $element->find('a', 0)->innertext,
'uri' => $element->find('a', 0)->href,
];
$html = getSimpleHTMLDOMCached($item['uri']);
$html = defaultLinkTo($html, parent::getURI());
$content = $html->find('#content .col-md', 1);
$info = explode(' by ', $content->find('p', 0)->plaintext);
$content->removeChild($content->firstChild());
$item['author'] = implode(' ', array_slice($info, 1));
$item['content'] = $content->innertext;
$this->items[] = $item;
$limit = $this->getInput('limit') ?? 15;
if ($this->getInput('feed') == 'posts') {
$this->collectExpandableDatas($this->getURI() . '.rss', $limit);
} else {
$this->collectEvent($this->getURI(), $limit);
}
}
private function collectEvent($html)
protected function parseItem(array $item)
{
$opt = [];
preg_match('/"csrfmiddlewaretoken" value="(.*)"/', $html, $opt);
$html = getSimpleHTMLDOMCached($item['uri']);
$html = defaultLinkTo($html, parent::getURI());
$item['categories'] = [$html->find('.breadcrumb-item', 1)->plaintext];
$content = $html->find('#content .col-md', 1);
$item['author'] = explode(' by ', $content->firstChild()->plaintext)[1];
$content->removeChild($content->firstChild());
$item['content'] = $content->innertext;
return $item;
}
private function collectEvent($uri, $limit)
{
$html = getSimpleHTMLDOMCached($uri);
preg_match('/"csrfmiddlewaretoken" value="(.*)"/', $html, $preg);
$header = [
"cookie: csrftoken=$opt[1]",
"x-csrftoken: $opt[1]",
"cookie: csrftoken=$preg[1]",
"x-csrftoken: $preg[1]",
'referer: ' . parent::getURI(),
];
preg_match('/var selected_feed = "(.*)";/', $html, $opt);
$post = [
'count' => $this->getInput('limit') ?? 20,
'feed' => $opt[1]
];
$opt = [ CURLOPT_POSTFIELDS => $post ];
preg_match('/var selected_feed = "(.*)";/', $html, $preg);
$opt = [ CURLOPT_POSTFIELDS => [
'count' => $limit,
'feed' => $preg[1]
]];
$html = getContents(parent::getURI() . 'events/_load_events', $header, $opt);
$html = defaultLinkTo(str_get_html($html), parent::getURI());
@ -83,10 +69,10 @@ class GovTrackBridge extends BridgeAbstract
foreach ($html->find('.tracked_event') as $event) {
$bill = $event->find('.event_title a, .event_body a', 0);
$date = explode(' ', $event->find('.event_date', 0)->plaintext);
preg_match('/Sponsor:(.*)\n/', $event->plaintext, $opt);
preg_match('/Sponsor:(.*)\n/', $event->plaintext, $preg);
$item = [
'author' => $opt[1] ?? '',
'author' => $preg[1] ?? '',
'content' => $event->find('td', 1)->innertext,
'enclosures' => [$event->find('img', 0)->src],
'timestamp' => strtotime(implode(' ', array_slice($date, 2))),
@ -115,10 +101,10 @@ class GovTrackBridge extends BridgeAbstract
public function getURI()
{
if ($this->getInput('feed') != 'posts') {
$url = parent::getURI() . 'events/' . $this->getInput('feed');
} else {
if ($this->getInput('feed') == 'posts') {
$url = parent::getURI() . $this->getInput('feed');
} else {
$url = parent::getURI() . 'events/' . $this->getInput('feed');
}
return $url;
}

View File

@ -47,6 +47,16 @@ Example: If the URL of the group displayed in the browser is :
https://www.hotukdeals.com/tag/broadband?sortBy=temp
Then enter :
broadband',
],
'subgroups' => [
'name' => 'category',
'type' => 'text',
'exampleValue' => '343563',
'title' => 'Category number in the URL : The category number that must be entered is present after "groups=" and before any "&".
Example: If the URL of the group displayed in the browser is :
https://www.hotukdeals.com/tag/broadband?groups=343563&sortBy=new
Then enter :
343563',
],
'order' => [
'name' => 'Order by',
@ -86,6 +96,7 @@ broadband',
'uri-group' => 'tag/',
'uri-deal' => 'deals/',
'uri-merchant' => 'search/deals?merchant-id=',
'image-host' => 'https://images.hotukdeals.com/',
'request-error' => 'Could not request HotUKDeals',
'thread-error' => 'Unable to determine the thread ID. Check the URL you entered',
'currency' => '£',

View File

@ -86,6 +86,11 @@ class InstagramBridge extends BridgeAbstract
$headers = [];
$sessionId = $this->getOption('session_id');
$dsUserId = $this->getOption('ds_user_id');
$headers[] = 'x-ig-app-id: 936619743392459';
$headers[] = 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36';
$headers[] = 'Accept-Language: en-US,en;q=0.9,ru;q=0.8';
$headers[] = 'Accept-Encoding: gzip, deflate, br';
$headers[] = 'Accept: */*';
if ($sessionId and $dsUserId) {
$headers[] = 'cookie: sessionid=' . $sessionId . '; ds_user_id=' . $dsUserId;
}
@ -125,8 +130,10 @@ class InstagramBridge extends BridgeAbstract
return;
}
if (!is_null($this->getInput('u'))) {
if (!is_null($this->getInput('u')) && !$this->fallbackMode) {
$userMedia = $data->data->user->edge_owner_to_timeline_media->edges;
} elseif (!is_null($this->getInput('u')) && $this->fallbackMode) {
$userMedia = $data->context->graphql_media;
} elseif (!is_null($this->getInput('h'))) {
$userMedia = $data->data->hashtag->edge_hashtag_to_media->edges;
} elseif (!is_null($this->getInput('l'))) {
@ -134,7 +141,12 @@ class InstagramBridge extends BridgeAbstract
}
foreach ($userMedia as $media) {
$media = $media->node;
// The media is not in the same element if in fallback mode than not
if (!$this->fallbackMode) {
$media = $media->node;
} else {
$media = $media->shortcode_media;
}
switch ($this->getInput('media_type')) {
case 'all':
@ -267,14 +279,39 @@ class InstagramBridge extends BridgeAbstract
protected function getInstagramJSON($uri)
{
// Sets fallbackMode to false
$this->fallbackMode = false;
if (!is_null($this->getInput('u'))) {
$userId = $this->getInstagramUserId($this->getInput('u'));
$data = $this->getContents(self::URI .
try {
$userId = $this->getInstagramUserId($this->getInput('u'));
$data = $this->getContents(self::URI .
'graphql/query/?query_hash=' .
self::USER_QUERY_HASH .
'&variables={"id"%3A"' .
$userId .
'"%2C"first"%3A10}');
} catch (HttpException $e) {
// If loading the data directly failed, we fall back to the "/embed" data loading
// We are in the fallback mode : set a booolean to handle this specific case while collecting the content
$this->fallbackMode = true;
// Get the HTML code of the profile embed page, and extract the JSON of it
$username = $this->getInput('u');
// Load the content using the integrated function to use helping headers
$htmlString = $this->getContents(self::URI . $username . '/embed/');
// Load the String as an SimpleHTMLDom Object
$html = new simple_html_dom();
$html->load($htmlString);
// Find the <script> tag containing the JSON content
$jsCode = $html->find('body', 0)->find('script', 3)->innertext;
// Extract the content needed by our bridge of the whole Javascript content
$regex = '#"contextJSON":"(.*)"}\]\],\["NavigationMetrics"#m';
preg_match($regex, $jsCode, $matches);
$jsVariable = $matches[1];
$data = stripcslashes($jsVariable);
// stripcslashes remove Javascript unicode escaping : add it back to the string so json_decode can handle it
$data = preg_replace('/(?<!\\\\)u[0-9A-Fa-f]{4}/', '\\\\$0', $data);
}
return json_decode($data);
} elseif (!is_null($this->getInput('h'))) {
$data = $this->getContents(self::URI .

View File

@ -0,0 +1,118 @@
<?php
class LeagueOfLegendsNewsBridge extends BridgeAbstract
{
const NAME = 'League of Legends News';
const URI = 'https://www.leagueoflegends.com';
const DESCRIPTION = 'Official League of Legends news.';
const MAINTAINER = 'KappaPrajd';
const PARAMETERS = [
[
'language' => [
'name' => 'Language',
'type' => 'list',
'defaultValue' => 'en-us',
'values' => [
'English (NA)' => 'en-us',
'English (EUW)' => 'en-gb',
'Deutsch' => 'de-de',
'Español (EUW)' => 'es-es',
'Français' => 'fr-fr',
'Italiano' => 'it-it',
'Polski' => 'pl-pl',
'Ελληνικά' => 'el-gr',
'Română' => 'ro-ro',
'Magyar' => 'hu-hu',
'Čeština' => 'cs-cz',
'Español (LATAM)' => 'es-mx',
'Português' => 'pt-br',
'日本語' => 'ja-jp',
'Русский' => 'ru-ru',
'Türkçe' => 'tr-tr',
'English (OCE)' => 'en-au',
'한국어' => 'ko-kr',
'English (SG)' => 'en-sg',
'English (PH)' => 'en-ph',
'Tiếng Việt' => 'vi-vn',
'ภาษาไทย' => 'th-th',
'繁體中文' => 'zh-tw',
'العربية' => 'ar-ae'
]
],
'category' => [
'name' => 'Category',
'type' => 'list',
'defaultValue' => 'all',
'values' => [
'All' => 'all',
'Game updates' => 'game-updates',
'Esports' => 'esports',
'Dev' => 'dev',
'Lore' => 'lore',
'Media' => 'media',
'Merch' => 'merch',
'Community' => 'community',
'Riot Games' => 'riot-games'
]
],
'onlyPatchNotes' => [
'name' => 'Only patch notes',
'type' => 'checkbox',
'defaultValue' => false,
],
],
];
public function collectData()
{
$siteUrl = $this->getSiteUrl();
$html = getSimpleHTMLDOM($siteUrl);
$articles = $html->find('a[data-testid=articlefeaturedcard-component]');
foreach ($articles as $article) {
$title = $article->find('div[data-testid=card-title]', 0)->plaintext;
$content = $article->find('div[data-testid=card-description] div div div', 0)->plaintext;
$timestamp = $article->find('div[data-testid=card-date] time', 0)->getAttribute('datetime');
$href = $article->getAttribute('href');
$item = [
'title' => $title,
'content' => $content,
'timestamp' => $timestamp,
'uri' => $this->getArticleUri($href),
];
$this->items[] = $item;
}
}
private function getSiteUrl()
{
$lang = $this->getInput('language');
$category = $this->getInput('category');
$onlyPatchNotes = $this->getInput('onlyPatchNotes');
$url = self::URI . '/' . $lang . '/news';
if ($onlyPatchNotes) {
return $url . '/tags/patch-notes';
} else if ($category === 'all') {
return $url;
}
return $url . '/' . $category;
}
private function getArticleUri($href)
{
$isInternalLink = str_starts_with($href, '/');
if ($isInternalLink) {
return self::URI . $href;
}
return $href;
}
}

View File

@ -78,8 +78,8 @@ class LfcPlBridge extends BridgeAbstract
foreach ($commentsDom as $comment) {
$header = $comment->find('.header', 0)->plaintext;
$content = $comment->find('.content', 0)->plaintext;
$comments .= $header . '<br />' . $content . '<br /><br />';
$commentContent = $comment->find('.content', 0)->plaintext;
$comments .= $header . '<br />' . $commentContent . '<br /><br />';
}
}
}

View File

@ -2,11 +2,11 @@
class ManyVidsBridge extends BridgeAbstract
{
const NAME = 'MANYVIDS';
const NAME = 'ManyVids';
const URI = 'https://www.manyvids.com';
const DESCRIPTION = 'Fetches the latest posts from a profile';
const MAINTAINER = 'dvikan';
const CACHE_TIMEOUT = 60 * 60;
const MAINTAINER = 'dvikan, subtle4553';
const CACHE_TIMEOUT = 3600;
const PARAMETERS = [
[
'profile' => [
@ -19,31 +19,103 @@ class ManyVidsBridge extends BridgeAbstract
]
];
private ?simple_html_dom $htmlDom = null;
private ?string $parsedProfileInput = null;
public function collectData()
{
$profile = $this->getInput('profile');
if (preg_match('#^(\d+/.*)$#', $profile, $m)) {
$profile = $m[1];
} elseif (preg_match('#https://www.manyvids.com/Profile/(\d+/\w+)#', $profile, $m)) {
$profile = $m[1];
} else {
throw new \Exception('nope');
if (!$profile) {
throw new \Exception('No value for \'profile\' was provided.');
}
$url = sprintf('https://www.manyvids.com/Profile/%s/Store/Videos/', $profile);
$dom = getSimpleHTMLDOM($url);
$videos = $dom->find('div[class^="ProfileTabGrid_card"]');
foreach ($videos as $item) {
$a = $item->find('a', 1);
$uri = 'https://www.manyvids.com' . $a->href;
if (preg_match('#Video/(\d+)/#', $uri, $m)) {
$uid = 'manyvids/' . $m[1];
if (preg_match('#^(\d+/.*)$#', $profile, $m)) {
$this->parsedProfileInput = $m[1];
} elseif (preg_match('#https://(www.)?manyvids.com/Profile/(\d+/.*?)/#', $profile, $m)) {
$this->parsedProfileInput = $m[2];
} else {
throw new \Exception(sprintf('Profile could not be parsed: %s', $profile));
}
$profileUrl = $this->getUri();
$url = sprintf('%s?sort=newest', $profileUrl);
$opt = [CURLOPT_COOKIE => 'sfwtoggle=false'];
$this->htmlDom = getSimpleHTMLDOM($url, [], $opt);
$elements = $this->htmlDom->find('div[class^="ProfileTabGrid_card__"]');
foreach ($elements as $element) {
$content = '';
$title = $element->find('span[class^="VideoCardUI_videoTitle__"] > a', 0);
if (!$title) {
continue;
}
$linkElement = $element->find('a[href^="/Video/"]', 0);
if ($linkElement) {
$itemUri = self::URI . $linkElement->getAttribute('href');
}
$image = $element->find('img', 0);
if ($image) {
if (isset($itemUri)) {
$content .= sprintf('<p><a href="%s"><img src="%s"></a></p>', $itemUri, $image->getAttribute('src'));
} else {
$content .= sprintf('<p><img src="%s"></p>', $image->getAttribute('src'));
}
}
$contentSegments = [];
$videoLength = $element->find('[class^="CardMedia_videoDuration__"] > span', 0);
if ($videoLength) {
$contentSegments[] = sprintf('%s', $videoLength->innertext);
}
$price = $element->find('[class^="PriceUI_regularPrice__"], [class^="PriceUI_card_price__"] > p, [class^="PriceUI_card_free_text__"]', 0);
$discountedPrice = $element->find('[class^="PriceUI_discountedPrice__"]', 0);
if ($price && $discountedPrice) {
$contentSegments[] = sprintf('<s>%s</s> <strong>%s</strong>', $price->innertext, $discountedPrice->innertext);
} elseif ($price && !$discountedPrice) {
$contentSegments[] = sprintf('<strong>%s</strong>', $price->innertext);
}
$content .= implode(' • ', $contentSegments);
$this->items[] = [
'title' => $a->plaintext,
'uri' => $uri,
'uid' => $uid ?? $uri,
'content' => $item->innertext,
'title' => $title->innertext,
'uri' => isset($itemUri) ? $itemUri : null,
'content' => $content,
];
}
}
public function getName()
{
if (!is_null($this->htmlDom)) {
$profileNameElement = $this->htmlDom->find('[class^="ProfileAboutMeUI_stageName__"]', 0);
if (!$profileNameElement) {
return parent::getName();
}
$profileNameElementContent = $profileNameElement->innertext;
$index = strpos($profileNameElementContent, '<');
$profileName = substr($profileNameElementContent, 0, $index);
return 'ManyVids: ' . $profileName;
}
return parent::getName();
}
public function getUri()
{
if (!is_null($this->parsedProfileInput)) {
return sprintf('%s/Profile/%s/Store/Videos', self::URI, $this->parsedProfileInput);
}
return parent::getUri();
}
}

View File

@ -0,0 +1,36 @@
<?php
class MinecraftBridge extends BridgeAbstract
{
const NAME = 'Minecraft';
const URI = 'https://www.minecraft.net';
const DESCRIPTION = 'Catch up on the latest Minecraft articles';
const MAINTAINER = 'tillcash';
public function getIcon()
{
return 'https://www.minecraft.net/etc.clientlibs/minecraftnet/clientlibs/clientlib-site/resources/favicon.ico';
}
public function collectData()
{
$json = getContents(
'https://www.minecraft.net/content/minecraftnet/language-masters/en-us/_jcr_content.articles.page-1.json'
);
$articles = json_decode($json);
if ($articles === null) {
returnServerError('Failed to decode JSON content.');
}
foreach ($articles->article_grid as $article) {
$this->items[] = [
'title' => $article->default_tile->title,
'uid' => $article->article_url,
'uri' => self::URI . $article->article_url,
'content' => $article->default_tile->sub_header,
];
}
}
}

View File

@ -48,6 +48,16 @@ https://www.mydealz.de/gruppe/dsl?sortBy=temp
Dann geben Sie ein:
dsl',
],
'subgroups' => [
'name' => 'Kategorie',
'type' => 'text',
'exampleValue' => '293',
'title' => 'Nummer des Kategorie in der URL: Der einzugebende Kategorienummer steht nach "groups=" und vor einem "&".
Beispiel: Wenn die URL der Gruppe, die im Browser angezeigt wird, :
https://www.mydealz.de/gruppe/telefon-internet?groups=153%2C154&sortBy=new&time_frame=0
Dann geben Sie ein:
153%2C154',
],
'order' => [
'name' => 'sortieren nach',
'type' => 'list',
@ -84,6 +94,7 @@ dsl',
'uri-group' => 'gruppe/',
'uri-deal' => 'deals/',
'uri-merchant' => 'search/gutscheine?merchant-id=',
'image-host' => 'https://static.mydealz.de/',
'request-error' => 'Could not request mydeals',
'thread-error' => 'Die ID der Diskussion kann nicht ermittelt werden. Überprüfen Sie die eingegebene URL',
'currency' => '€',

View File

@ -6,7 +6,7 @@ class NurembergerNachrichtenBridge extends BridgeAbstract
const NAME = 'Nürnberger Nachrichten';
const CACHE_TIMEOUT = 3600;
const URI = 'https://www.nn.de';
const DESCRIPTION = 'Bridge for Bavarian regional news site nordbayern.de';
const DESCRIPTION = 'Bridge for NurembergerNachrichten news site nn.de';
const PARAMETERS = [ [
'region' => [
'name' => 'region',
@ -66,7 +66,7 @@ class NurembergerNachrichtenBridge extends BridgeAbstract
// exclude nn+ articles if desired
if (
$this->getInput('hideNNPlus') &&
str_contains($articleContent->find('article[id=article]', 0)->find('header', 0), 'icon-nnplus')
$articleContent->find('div[class=paywall]')
) {
continue;
}

View File

@ -14,8 +14,8 @@ class OpenCVEBridge extends BridgeAbstract
'instance' => [
'name' => 'OpenCVE Instance',
'required' => true,
'defaultValue' => 'https://www.opencve.io',
'exampleValue' => 'https://www.opencve.io'
'defaultValue' => 'https://app.opencve.io',
'exampleValue' => 'https://app.opencve.io'
],
'login' => [
'name' => 'Login',
@ -155,14 +155,14 @@ class OpenCVEBridge extends BridgeAbstract
$titlePrefix = '[' . $queryName . '] ';
}
foreach (json_decode($response) as $cveItem) {
if (array_key_exists($cveItem->id, $fetchedIds)) {
foreach (json_decode($response)->results as $cveItem) {
if (array_key_exists($cveItem->cve_id, $fetchedIds)) {
continue;
}
$fetchedIds[$cveItem->id] = true;
$fetchedIds[$cveItem->cve_id] = true;
$item = [
'uri' => $instance . '/cve/' . $cveItem->id,
'uid' => $cveItem->id,
'uri' => $instance . '/cve/' . $cveItem->cve_id,
'uid' => $cveItem->cve_id,
];
if ($this->getInput('upd_timestamp') == 1) {
$item['timestamp'] = strtotime($cveItem->updated_at);
@ -179,7 +179,7 @@ class OpenCVEBridge extends BridgeAbstract
$item['content'] = $content;
$item['title'] = $title;
} else {
$item['content'] = $cveItem->summary . $this->getLinks($cveItem->id);
$item['content'] = $cveItem->description . $this->getLinks($cveItem->cve_id);
$item['title'] = $this->getTitle($titlePrefix, $cveItem);
}
$this->items[] = $item;
@ -193,17 +193,17 @@ class OpenCVEBridge extends BridgeAbstract
private function getTitle($titlePrefix, $cveItem)
{
$summary = $cveItem->summary;
$summary = $cveItem->description;
$limit = $this->getInput('limit');
if ($limit && mb_strlen($summary) > 100) {
$summary = mb_substr($summary, 0, $limit) + '...';
}
return $titlePrefix . $cveItem->id . '. ' . $summary;
return $titlePrefix . $cveItem->cve_id . '. ' . $summary;
}
private function fetchContents($cveItem, $titlePrefix, $instance, $authHeader)
{
$url = $instance . '/api/cve/' . $cveItem->id;
$url = $instance . '/api/cve/' . $cveItem->cve_id;
$response = getContents($url, [$authHeader]);
$datum = json_decode($response);
@ -211,26 +211,36 @@ class OpenCVEBridge extends BridgeAbstract
$title = $this->getTitleFromDatum($datum, $titlePrefix);
$result = self::CSS;
$result .= '<h1>' . $cveItem->id . '</h1>';
$result .= '<h1>' . $cveItem->cve_id . '</h1>';
$result .= $this->getCVSSLabels($datum);
$result .= '<p>' . $datum->summary . '</p>';
$result .= '<p>' . $datum->description . '</p>';
$result .= <<<EOD
<h3>Information:</h3>
<p>
<ul>
<li><b>Publication date</b>: {$datum->raw_nvd_data->published}
<li><b>Last modified</b>: {$datum->raw_nvd_data->lastModified}
<li><b>Last modified</b>: {$datum->raw_nvd_data->lastModified}
<li><b>Created At</b>: {$datum->created_at}
<li><b>Updated At</b>: {$datum->updated_at}
</ul>
</p>
EOD;
$result .= $this->getV3Table($datum);
$result .= $this->getV2Table($datum);
if (isset($datum->metrics->cvssV4_0->data->vector)) {
$result .= $this->cvssV4VectorToTable($datum->metrics->cvssV4_0->data->vector);
}
$result .= $this->getLinks($datum->id);
$result .= $this->getReferences($datum);
if (isset($datum->metrics->cvssV3_1->data->vector)) {
$result .= $this->cvssV3VectorToTable($datum->metrics->cvssV3_1->data->vector);
}
if (isset($datum->metrics->cvssV3_0->data->vector)) {
$result .= $this->cvssV3VectorToTable($datum->metrics->cvssV3_0->data->vector);
}
if (isset($datum->metrics->cvssV2_0->data->vector)) {
$result .= $this->cvssV2VectorToTable($datum->metrics->cvssV2_0->data->vector);
}
$result .= $this->getLinks($datum->cve_id);
$result .= $this->getVendors($datum);
return [$result, $title];
@ -239,14 +249,20 @@ class OpenCVEBridge extends BridgeAbstract
private function getTitleFromDatum($datum, $titlePrefix)
{
$title = $titlePrefix;
if ($datum->cvss->v3) {
$title .= "[v3: {$datum->cvss->v3}] ";
if (isset($datum->metrics->cvssV4_0->data->score)) {
$title .= "[v4: {$datum->metrics->cvssV4_0->data->score}] ";
}
if ($datum->cvss->v2) {
$title .= "[v2: {$datum->cvss->v2}] ";
if (isset($datum->metrics->cvssV3_1->data->score)) {
$title .= "[v3.1: {$datum->metrics->cvssV3_1->data->score}] ";
}
$title .= $datum->id . '. ';
$titlePostfix = $datum->summary;
if (isset($datum->metrics->cvssV3_0->data->score)) {
$title .= "[v3: {$datum->metrics->cvssV3_0->data->score}] ";
}
if (isset($datum->metrics->cvssV2_0->data->score)) {
$title .= "[v2: {$datum->metrics->cvssV2_0->data->score}] ";
}
$title .= $datum->cve_id . '. ';
$titlePostfix = $datum->description;
$limit = $this->getInput('limit');
if ($limit && mb_strlen($titlePostfix) > 100) {
$titlePostfix = mb_substr($titlePostfix, 0, $limit) + '...';
@ -257,64 +273,49 @@ class OpenCVEBridge extends BridgeAbstract
private function getCVSSLabels($datum)
{
$CVSSv2Text = 'n/a';
$CVSSv2Class = 'cvss-na-color';
if ($datum->cvss->v2) {
$importance = '';
if ($datum->cvss->v2 >= 7) {
$importance = 'HIGH';
$CVSSv2Class = 'cvss-high-color';
} else if ($datum->cvss->v2 >= 4) {
$importance = 'MEDIUM';
$CVSSv2Class = 'cvss-medium-color';
} else {
$importance = 'LOW';
$CVSSv2Class = 'cvss-low-color';
}
$CVSSv2Text = sprintf('[%s] %.1f', $importance, $datum->cvss->v2);
$cvss4 = '';
$cvss31 = '';
$cvss3 = '';
$cvss2 = '';
if (isset($datum->metrics->cvssV4_0->data->score)) {
$cvss4 = $this->formatCVSSLabel($datum->metrics->cvssV4_0->data->score, '4.0', 9, 7, 4);
}
if (isset($datum->metrics->cvssV3_1->data->score)) {
$cvss31 = $this->formatCVSSLabel($datum->metrics->cvssV3_1->data->score, '3.1', 9, 7, 4);
}
if (isset($datum->metrics->cvssV3_0->data->score)) {
$cvss3 = $this->formatCVSSLabel($datum->metrics->cvssV3_0->data->score, '3.0', 9, 7, 4);
}
if (isset($datum->metrics->cvssV2_0->data->score)) {
$cvss2 = $this->formatCVSSLabel($datum->metrics->cvssV2_0->data->score, '2.0', 99, 7, 4);
}
$CVSSv2Item = "<div>CVSS v2: </div><div class=\"label {$CVSSv2Class}\">{$CVSSv2Text}</div>";
$CVSSv3Text = 'n/a';
$CVSSv3Class = 'cvss-na-color';
if ($datum->cvss->v3) {
$importance = '';
if ($datum->cvss->v3 >= 9) {
$importance = 'CRITICAL';
$CVSSv3Class = 'cvss-crit-color';
} else if ($datum->cvss->v3 >= 7) {
$importance = 'HIGH';
$CVSSv3Class = 'cvss-high-color';
} else if ($datum->cvss->v3 >= 4) {
$importance = 'MEDIUM';
$CVSSv3Class = 'cvss-medium-color';
} else {
$importance = 'LOW';
$CVSSv3Class = 'cvss-low-color';
}
$CVSSv3Text = sprintf('[%s] %.1f', $importance, $datum->cvss->v3);
}
$CVSSv3Item = "<div>CVSS v3: </div><div class=\"label {$CVSSv3Class}\">{$CVSSv3Text}</div>";
return '<div class="labels-row">' . $CVSSv3Item . $CVSSv2Item . '</div>';
return '<div class="labels-row">' . $cvss4 . $cvss31 . $cvss3 . $cvss2 . '</div>';
}
private function getReferences($datum)
private function formatCVSSLabel($score, $version, $critical_thr, $high_thr, $medium_thr)
{
if (count($datum->raw_nvd_data->references) == 0) {
return '';
}
$res = '<h3>References:</h3> <p><ul>';
foreach ($datum->raw_nvd_data->references as $ref) {
$item = '<li>';
if (isset($ref->tags) && count($ref->tags) > 0) {
$item .= '[' . implode(', ', $ref->tags) . '] ';
$text = 'n/a';
$class = 'cvss-na-color';
if ($score) {
$importance = '';
if ($score >= $critical_thr) {
$importance = 'CRITICAL';
$class = 'cvss-crit-color';
} else if ($score >= $high_thr) {
$importance = 'HIGH';
$class = 'cvss-high-color';
} else if ($score >= $medium_thr) {
$importance = 'MEDIUM';
$class = 'cvss-medium-color';
} else {
$importance = 'LOW';
$class = 'cvss-low-color';
}
$item .= "<a href=\"{$ref->url}\">{$ref->url}</a>";
$item .= '<li>';
$res .= $item;
$text = sprintf('[%s] %.1f', $importance, $score);
}
$res .= '</p></ul>';
return $res;
$item = "<div>CVSS {$version}: </div><div class=\"label {$class}\">{$text}</div>";
return $item;
}
private function getLinks($id)
@ -331,84 +332,253 @@ class OpenCVEBridge extends BridgeAbstract
EOD;
}
private function getV3Table($datum)
private function cvssV3VectorToTable($cvssVector)
{
$metrics = $datum->raw_nvd_data->metrics;
if (!isset($metrics->cvssMetricV31) || count($metrics->cvssMetricV31) == 0) {
return '';
$vectorComponents = [];
$parts = explode('/', $cvssVector);
if (!preg_match('/^CVSS:3\.[01]/', $parts[0])) {
return 'Error: Not a valid CVSS v3.0 or v3.1 vector';
}
$v3 = $metrics->cvssMetricV31[0];
$data = $v3->cvssData;
return <<<EOD
<div class="cvss-table">
for ($i = 1; $i < count($parts); $i++) {
$component = explode(':', $parts[$i]);
if (count($component) == 2) {
$vectorComponents[$component[0]] = $component[1];
}
}
$readableNames = [
'AV' => ['N' => 'Network', 'A' => 'Adjacent', 'L' => 'Local', 'P' => 'Physical'],
'AC' => ['L' => 'Low', 'H' => 'High'],
'PR' => ['N' => 'None', 'L' => 'Low', 'H' => 'High'],
'UI' => ['N' => 'None', 'R' => 'Required'],
'S' => ['U' => 'Unchanged', 'C' => 'Changed'],
'C' => ['N' => 'None', 'L' => 'Low', 'H' => 'High'],
'I' => ['N' => 'None', 'L' => 'Low', 'H' => 'High'],
'A' => ['N' => 'None', 'L' => 'Low', 'H' => 'High']
];
$data = new stdClass();
$data->attackVector = isset($readableNames['AV'][$vectorComponents['AV']]) ? $readableNames['AV'][$vectorComponents['AV']] : 'Unknown';
$data->attackComplexity = isset($readableNames['AC'][$vectorComponents['AC']]) ? $readableNames['AC'][$vectorComponents['AC']] : 'Unknown';
$data->privilegesRequired = isset($readableNames['PR'][$vectorComponents['PR']]) ? $readableNames['PR'][$vectorComponents['PR']] : 'Unknown';
$data->userInteraction = isset($readableNames['UI'][$vectorComponents['UI']]) ? $readableNames['UI'][$vectorComponents['UI']] : 'Unknown';
$data->scope = isset($readableNames['S'][$vectorComponents['S']]) ? $readableNames['S'][$vectorComponents['S']] : 'Unknown';
$data->confidentialityImpact = isset($readableNames['C'][$vectorComponents['C']]) ? $readableNames['C'][$vectorComponents['C']] : 'Unknown';
$data->integrityImpact = isset($readableNames['I'][$vectorComponents['I']]) ? $readableNames['I'][$vectorComponents['I']] : 'Unknown';
$data->availabilityImpact = isset($readableNames['A'][$vectorComponents['A']]) ? $readableNames['A'][$vectorComponents['A']] : 'Unknown';
$html = '<div class="cvss-table">
<h3>CVSS v3 details</h3>
<table>
<tr>
<td>Impact score</td><td>{$v3->impactScore}</td>
<td>Exploitability score</td><td>{$v3->exploitabilityScore}</td>
<td>Attack vector</td><td>' . $data->attackVector . '</td>
<td>Confidentiality Impact</td><td>' . $data->confidentialityImpact . '</td>
</tr>
<tr>
<td>Attack vector</td><td>{$data->attackVector}</td>
<td>Confidentiality Impact</td><td>{$data->confidentialityImpact}</td>
<td>Attack complexity</td><td>' . $data->attackComplexity . '</td>
<td>Integrity Impact</td><td>' . $data->integrityImpact . '</td>
</tr>
<tr>
<td>Attack complexity</td><td>{$data->attackComplexity}</td>
<td>Integrity Impact</td><td>{$data->integrityImpact}</td>
<td>Privileges Required</td><td>' . $data->privilegesRequired . '</td>
<td>Availability Impact</td><td>' . $data->availabilityImpact . '</td>
</tr>
<tr>
<td>Privileges Required</td><td>{$data->privilegesRequired}</td>
<td>Availability Impact</td><td>{$data->availabilityImpact}</td>
</tr>
<tr>
<td>User Interaction</td><td>{$data->userInteraction}</td>
<td>Scope</td><td>{$data->scope}</td>
<td>User Interaction</td><td>' . $data->userInteraction . '</td>
<td>Scope</td><td>' . $data->scope . '</td>
</tr>
</table>
</div>
EOD;
</div>';
return $html;
}
private function getV2Table($datum)
private function cvssV2VectorToTable($cvssVector)
{
$metrics = $datum->raw_nvd_data->metrics;
if (!isset($metrics->cvssMetricV2) || count($metrics->cvssMetricV2) == 0) {
return '';
$vectorComponents = [];
$parts = explode('/', $cvssVector);
foreach ($parts as $part) {
$component = explode(':', $part);
if (count($component) == 2) {
$vectorComponents[$component[0]] = $component[1];
}
}
$v2 = $metrics->cvssMetricV2[0];
$data = $v2->cvssData;
return <<<EOD
<div class="cvss-table">
$readableNames = [
'AV' => ['L' => 'Local', 'A' => 'Adjacent Network', 'N' => 'Network'],
'AC' => ['H' => 'High', 'M' => 'Medium', 'L' => 'Low'],
'Au' => ['M' => 'Multiple', 'S' => 'Single', 'N' => 'None'],
'C' => ['N' => 'None', 'P' => 'Partial', 'C' => 'Complete'],
'I' => ['N' => 'None', 'P' => 'Partial', 'C' => 'Complete'],
'A' => ['N' => 'None', 'P' => 'Partial', 'C' => 'Complete']
];
$metricValues = [
'AV' => ['L' => 0.395, 'A' => 0.646, 'N' => 1.0],
'AC' => ['H' => 0.35, 'M' => 0.61, 'L' => 0.71],
'Au' => ['M' => 0.45, 'S' => 0.56, 'N' => 0.704],
'C' => ['N' => 0, 'P' => 0.275, 'C' => 0.660],
'I' => ['N' => 0, 'P' => 0.275, 'C' => 0.660],
'A' => ['N' => 0, 'P' => 0.275, 'C' => 0.660]
];
$confImpact = isset($metricValues['C'][$vectorComponents['C']]) ? $metricValues['C'][$vectorComponents['C']] : 0;
$integImpact = isset($metricValues['I'][$vectorComponents['I']]) ? $metricValues['I'][$vectorComponents['I']] : 0;
$availImpact = isset($metricValues['A'][$vectorComponents['A']]) ? $metricValues['A'][$vectorComponents['A']] : 0;
$impact = 10.41 * (1 - (1 - $confImpact) * (1 - $integImpact) * (1 - $availImpact));
$av = isset($metricValues['AV'][$vectorComponents['AV']]) ? $metricValues['AV'][$vectorComponents['AV']] : 0;
$ac = isset($metricValues['AC'][$vectorComponents['AC']]) ? $metricValues['AC'][$vectorComponents['AC']] : 0;
$au = isset($metricValues['Au'][$vectorComponents['Au']]) ? $metricValues['Au'][$vectorComponents['Au']] : 0;
$exploitability = 20 * $av * $ac * $au;
$impact = round($impact, 1);
$exploitability = round($exploitability, 1);
$data = new stdClass();
$data->accessVector = isset($readableNames['AV'][$vectorComponents['AV']]) ? $readableNames['AV'][$vectorComponents['AV']] : 'Unknown';
$data->accessComplexity = isset($readableNames['AC'][$vectorComponents['AC']]) ? $readableNames['AC'][$vectorComponents['AC']] : 'Unknown';
$data->authentication = isset($readableNames['Au'][$vectorComponents['Au']]) ? $readableNames['Au'][$vectorComponents['Au']] : 'Unknown';
$data->confidentialityImpact = isset($readableNames['C'][$vectorComponents['C']]) ? $readableNames['C'][$vectorComponents['C']] : 'Unknown';
$data->integrityImpact = isset($readableNames['I'][$vectorComponents['I']]) ? $readableNames['I'][$vectorComponents['I']] : 'Unknown';
$data->availabilityImpact = isset($readableNames['A'][$vectorComponents['A']]) ? $readableNames['A'][$vectorComponents['A']] : 'Unknown';
$v2 = new stdClass();
$v2->impactScore = $impact;
$v2->exploitabilityScore = $exploitability;
$html = '<div class="cvss-table">
<h3>CVSS v2 details</h3>
<table>
<tr>
<td>Impact score</td><td>{$v2->impactScore}</td>
<td>Exploitability score</td><td>{$v2->exploitabilityScore}</td>
<td>Impact score</td><td>' . $v2->impactScore . '</td>
<td>Exploitability score</td><td>' . $v2->exploitabilityScore . '</td>
</tr>
<tr>
<td>Access Vector</td><td>{$data->accessVector}</td>
<td>Confidentiality Impact</td><td>{$data->confidentialityImpact}</td>
<td>Access Vector</td><td>' . $data->accessVector . '</td>
<td>Confidentiality Impact</td><td>' . $data->confidentialityImpact . '</td>
</tr>
<tr>
<td>Access Complexity</td><td>{$data->accessComplexity}</td>
<td>Integrity Impact</td><td>{$data->integrityImpact}</td>
<td>Access Complexity</td><td>' . $data->accessComplexity . '</td>
<td>Integrity Impact</td><td>' . $data->integrityImpact . '</td>
</tr>
<tr>
<td>Authentication</td><td>{$data->authentication}</td>
<td>Availability Impact</td><td>{$data->availabilityImpact}</td>
<td>Authentication</td><td>' . $data->authentication . '</td>
<td>Availability Impact</td><td>' . $data->availabilityImpact . '</td>
</tr>
<tr>
</table>
</div>
EOD;
</div>';
return $html;
}
private function cvssV4VectorToTable($cvssVector)
{
$vectorComponents = [];
$parts = explode('/', $cvssVector);
if (!preg_match('/^CVSS:4\.0/', $parts[0])) {
return 'Error: Not a valid CVSS v4.0 vector';
}
for ($i = 1; $i < count($parts); $i++) {
$component = explode(':', $parts[$i]);
if (count($component) == 2) {
$vectorComponents[$component[0]] = $component[1];
}
}
$readableNames = [
'AV' => ['N' => 'Network', 'A' => 'Adjacent', 'L' => 'Local', 'P' => 'Physical'],
'AC' => ['L' => 'Low', 'H' => 'High'],
'AT' => ['N' => 'None', 'P' => 'Present'],
'PR' => ['N' => 'None', 'L' => 'Low', 'H' => 'High'],
'UI' => ['N' => 'None', 'P' => 'Passive', 'A' => 'Active'],
'VC' => ['N' => 'None', 'L' => 'Low', 'H' => 'High'],
'VI' => ['N' => 'None', 'L' => 'Low', 'H' => 'High'],
'VA' => ['N' => 'None', 'L' => 'Low', 'H' => 'High'],
'SC' => ['N' => 'None', 'L' => 'Low', 'H' => 'High'],
'SI' => ['N' => 'None', 'L' => 'Low', 'H' => 'High'],
'SA' => ['N' => 'None', 'L' => 'Low', 'H' => 'High']
];
$data = new stdClass();
$data->attackVector = isset($readableNames['AV'][$vectorComponents['AV']]) ? $readableNames['AV'][$vectorComponents['AV']] : 'Unknown';
$data->attackComplexity = isset($readableNames['AC'][$vectorComponents['AC']]) ? $readableNames['AC'][$vectorComponents['AC']] : 'Unknown';
$data->privilegesRequired = isset($readableNames['PR'][$vectorComponents['PR']]) ? $readableNames['PR'][$vectorComponents['PR']] : 'Unknown';
$data->attackRequirements = isset($readableNames['AT'][$vectorComponents['AT']]) ? $readableNames['AT'][$vectorComponents['AT']] : 'Unknown';
$data->userInteraction = isset($readableNames['UI'][$vectorComponents['UI']]) ? $readableNames['UI'][$vectorComponents['UI']] : 'Unknown';
$data->confidentialityImpact = isset($readableNames['VC'][$vectorComponents['VC']]) ? $readableNames['VC'][$vectorComponents['VC']] : 'Unknown';
$data->integrityImpact = isset($readableNames['VI'][$vectorComponents['VI']]) ? $readableNames['VI'][$vectorComponents['VI']] : 'Unknown';
$data->availabilityImpact = isset($readableNames['VA'][$vectorComponents['VA']]) ? $readableNames['VA'][$vectorComponents['VA']] : 'Unknown';
$data->confidentialityImpactS = isset($readableNames['SC'][$vectorComponents['SC']]) ? $readableNames['SC'][$vectorComponents['SC']] : 'Unknown';
$data->integrityImpactS = isset($readableNames['SI'][$vectorComponents['SI']]) ? $readableNames['SI'][$vectorComponents['SI']] : 'Unknown';
$data->availabilityImpactS = isset($readableNames['SA'][$vectorComponents['SA']]) ? $readableNames['SA'][$vectorComponents['SA']] : 'Unknown';
$html = '<div class="cvss-table">
<h3>CVSS v4.0 details</h3>
<table>
<tr>
<td>Attack vector</td><td>' . $data->attackVector . '</td>
<td>Vulnerable System Confidentiality Impact</td><td>' . $data->confidentialityImpact . '</td>
</tr>
<tr>
<td>Attack complexity</td><td>' . $data->attackComplexity . '</td>
<td>Vulnerable System Integrity Impact</td><td>' . $data->integrityImpact . '</td>
</tr>
<tr>
<td>Privileges Required</td><td>' . $data->privilegesRequired . '</td>
<td>Vulnerable System Availability Impact</td><td>' . $data->availabilityImpact . '</td>
</tr>
<tr>
<td>Attack Requirements</td><td>' . $data->attackRequirements . '</td>
<td>Subsequent System Confidentiality Impact</td><td>' . $data->confidentialityImpactS . '</td>
</tr>
<tr>
<td>User Interaction</td><td>' . $data->userInteraction . '</td>
<td>Subsequent System Integrity Impact</td><td>' . $data->integrityImpactS . '</td>
</tr>
<tr>
<td></td><td></td>
<td>Subsequent System Avaliablity Impact</td><td>' . $data->availabilityImpactS . '</td>
</tr>
</table>
</div>';
return $html;
}
private function getVendors($datum)
{
if (count((array)$datum->vendors) == 0) {
return '';
}
$vendor_data = [];
foreach ($datum->vendors as $vendor_str) {
$pieces = explode('$PRODUCT$', $vendor_str);
if (count($pieces) == 1) {
$vendor = $pieces[0];
if (!array_key_exists($vendor, $vendor_data)) {
$vendor_data[$vendor] = [];
}
} else {
$vendor = $pieces[0];
$product = $pieces[1];
if (!array_key_exists($vendor, $vendor_data)) {
$vendor_data[$vendor] = [];
}
array_push($vendor_data[$vendor], $product);
}
}
$res = '<h3>Affected products</h3><p><ul>';
foreach ($datum->vendors as $vendor => $products) {
foreach ($vendor_data as $vendor => $products) {
$res .= "<li>{$vendor}";
if (count($products) > 0) {
$res .= '<ul>';
@ -420,5 +590,6 @@ class OpenCVEBridge extends BridgeAbstract
$res .= '</li>';
}
$res .= '</ul></p>';
return $res;
}
}

View File

@ -62,7 +62,7 @@ class PepperBridgeAbstract extends BridgeAbstract
foreach ($list as $deal) {
// Get the JSON Data stored as vue
$jsonDealData = $this->getDealJsonData($deal);
$dealMeta = Json::decode($deal->find('div[class=threadGrid-headerMeta]', 0)->find('div[class=js-vue2]', 1)->getAttribute('data-vue2'));
$dealMeta = Json::decode($deal->find('div[class=js-vue2]', 1)->getAttribute('data-vue2'));
$item = [];
$item['uri'] = $this->getDealURI($jsonDealData);
@ -80,7 +80,7 @@ class PepperBridgeAbstract extends BridgeAbstract
. $this->getShipsFrom($dealMeta)
. $this->getShippingCost($jsonDealData)
. $this->getSource($jsonDealData)
. $this->getDealLocation($dealMeta)
. $this->getDealLocation($jsonDealData)
. $deal->find('div[class*=' . $selectorDescription . ']', 0)->innertext
. '</td><td>'
. $this->getTemperature($jsonDealData)
@ -402,14 +402,9 @@ HEREDOC;
* Get the Deal location if it exists
* @return string String of the deal location
*/
private function getDealLocation($dealMeta)
private function getDealLocation($jsonDealData)
{
$ribbons = $dealMeta['props']['metaRibbons'];
$isLocal = false;
foreach ($ribbons as $ribbon) {
$isLocal |= ($ribbon['type'] == 'local');
}
if ($isLocal) {
if ($jsonDealData['props']['thread']['isLocal']) {
$content = '<div>' . $this->i8n('deal-type') . ' : ' . $this->i8n('localdeal') . '</div>';
} else {
$content = '';
@ -424,8 +419,11 @@ HEREDOC;
private function getImage($deal)
{
// Get thread Image JSON content
$content = Json::decode($deal->find('div[class*=threadGrid-image]', 0)->find('div[class=js-vue2]', 0)->getAttribute('data-vue2'));
return '<img src="' . $content['props']['threadImageUrl'] . '"/>';
$content = Json::decode($deal->find('div[class=js-vue2]', 0)->getAttribute('data-vue2'));
//return '<img src="' . $content['props']['threadImageUrl'] . '"/>';
return '<img src="' . $this->i8n('image-host') . $content['props']['thread']['mainImage']['path'] . '/'
. $content['props']['thread']['mainImage']['name'] . '/re/202x202/qt/70/'
. $content['props']['thread']['mainImage']['uid'] . '"/>';
}
/**
@ -434,7 +432,7 @@ HEREDOC;
*/
private function getShipsFrom($dealMeta)
{
$metas = $dealMeta['props']['metaRibbons'];
$metas = $dealMeta['props']['metaRibbons'] ?? [];
$shipsFrom = null;
foreach ($metas as $meta) {
if ($meta['type'] == 'dispatched-from') {
@ -524,6 +522,7 @@ HEREDOC;
{
$group = $this->getInput('group');
$order = $this->getInput('order');
$subgroups = $this->getInput('subgroups');
// This permit to keep the existing Feed to work
if ($order == $this->i8n('context-hot')) {
@ -533,7 +532,7 @@ HEREDOC;
}
$url = $this->i8n('bridge-uri')
. $this->i8n('uri-group') . $group . '?sortBy=' . $sortBy;
. $this->i8n('uri-group') . $group . '?sortBy=' . $sortBy . '&groups=' . $subgroups;
return $url;
}

View File

@ -40,7 +40,7 @@ class RadioMelodieBridge extends BridgeAbstract
$picture = [];
// Get the Main picture URL
$picture[] = self::URI . $article->find('figure[class*=photoviewer]', 0)->find('img', 0)->src;
$picture[] = $article->find('figure[class*=photoviewer]', 0)->find('img', 0)->src;
$audioHTML = $article->find('audio');
// Add the audio element to the enclosure
@ -123,7 +123,7 @@ class RadioMelodieBridge extends BridgeAbstract
preg_match('/wavesurfer[0-9]+.load\(\'(.*)\'\)/m', $js->innertext, $urls);
// Create the plain HTML <audio> content to play this audio file
$content = '<audio style="width: 100%" src="' . $urls[1] . '" controls ></audio>';
$content = '<audio style="width: 100%" src="' . self::URI . $urls[1] . '" controls ></audio>';
// Replace the <script> tag by the <audio> tag
$js->outertext = $content;

View File

@ -20,7 +20,15 @@ class RedditBridge extends BridgeAbstract
'required' => false,
'type' => 'number',
'exampleValue' => 100,
'title' => 'Filter out posts with lower score'
'title' => 'Filter out posts with lower score. Set to -1 to disable. If both score and comments are set, an OR is applied.',
],
'min_comments' => [
'name' => 'Minimal number of comments',
'required' => false,
'type' => 'number',
'exampleValue' => 100,
'title' => 'Filter out posts with lower number of comments. Set to -1 to disable. If both score and comments are set, an OR is applied.',
'defaultValue' => -1
],
'd' => [
'name' => 'Sort By',
@ -30,10 +38,25 @@ class RedditBridge extends BridgeAbstract
'Hot' => 'hot',
'Relevance' => 'relevance',
'New' => 'new',
'Top' => 'top'
'Top' => 'top',
'Comments' => 'comments',
],
'defaultValue' => 'Hot'
],
't' => [
'name' => 'Time',
'type' => 'list',
'title' => 'Sort by new, hot, top or relevancy',
'values' => [
'All' => 'all',
'Year' => 'year',
'Month' => 'month',
'Week' => 'week',
'Day' => 'day',
'Hour' => 'hour',
],
'defaultValue' => 'week'
],
'search' => [
'name' => 'Keyword search',
'required' => false,
@ -126,6 +149,7 @@ class RedditBridge extends BridgeAbstract
$frontend = 'https://old.reddit.com';
}
$section = $this->getInput('d');
$time = $this->getInput('t');
switch ($this->queriedContext) {
case 'single':
@ -147,7 +171,7 @@ class RedditBridge extends BridgeAbstract
foreach ($subreddits as $subreddit) {
$version = 'v0.0.2';
$useragent = "rss-bridge $version (https://github.com/RSS-Bridge/rss-bridge)";
$url = self::createUrl($search, $flareInput, $subreddit, $user, $section, $this->queriedContext);
$url = self::createUrl($search, $flareInput, $subreddit, $user, $section, $time, $this->queriedContext);
$response = getContents($url, ['User-Agent: ' . $useragent], [], true);
@ -162,8 +186,20 @@ class RedditBridge extends BridgeAbstract
$data = $post->data;
if ($data->score < $this->getInput('score')) {
continue;
$min_score = $this->getInput('score');
$min_comments = $this->getInput('min_comments');
if ($min_score >= 0 && $min_comments >= 0) {
if ($data->num_comments < $min_comments || $data->score < $min_score) {
continue;
}
} elseif ($min_score >= 0) {
if ($data->score < $min_score) {
continue;
}
} elseif ($min_comments >= 0) {
if ($data->num_comments < $min_comments) {
continue;
}
}
$item = [];
@ -264,7 +300,7 @@ class RedditBridge extends BridgeAbstract
});
}
public static function createUrl($search, $flareInput, $subreddit, bool $user, $section, $queriedContext): string
public static function createUrl($search, $flareInput, $subreddit, bool $user, $section, $time, $queriedContext): string
{
if ($search === '') {
$keywords = '';
@ -286,6 +322,7 @@ class RedditBridge extends BridgeAbstract
'q' => $keywords . $flair . ($user ? 'author:' : 'subreddit:') . $name,
'sort' => $section,
'include_over_18' => 'on',
't' => $time
];
return 'https://old.reddit.com/search.json?' . http_build_query($query);
}

View File

@ -35,7 +35,7 @@ class ReutersBridge extends BridgeAbstract
'title' => 'Feeds from Reuters U.S/International edition',
'values' => [
'Top News' => 'home/topnews',
'Fact Check' => 'chan:abtpk0vm',
'Fact Check' => '/fact-check',
'Entertainment' => 'chan:8ym8q8dl',
'Politics' => 'politics',
'Wire' => 'wire',
@ -137,7 +137,6 @@ class ReutersBridge extends BridgeAbstract
const OLD_WIRE_SECTION = [
'home/topnews',
'chan:abtpk0vm',
'chan:8ym8q8dl',
'politics',
'wire'

100
bridges/ShadertoyBridge.php Normal file
View File

@ -0,0 +1,100 @@
<?php
class ShadertoyBridge extends BridgeAbstract
{
const NAME = 'Shadertoy';
const URI = 'https://www.shadertoy.com';
const DESCRIPTION = 'Latest submissions on Shadertoy';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
const PARAMETERS = [
[
'category' => [
'name' => 'category',
'type' => 'list',
'exampleValue' => 'Popular',
'title' => 'Select a category',
'values' => [
'Shaders of the Week' => 'sotw',
'Popular' => 'popular',
'Newest' => 'newest',
'Hot' => 'hot',
]
]
]
];
public function postprocessDescription($content)
{
// replace [url] tags
$pattern = '/\[\/?url.*?\]/';
$replace = '';
$content = preg_replace($pattern, $replace, $content);
// find URLs and turn then into hyperlinks
$pattern = '/(http|https|ftp|ftps)\:\/\/[a-zA-Z0-9\-\.]+\.[a-zA-Z]{2,3}(\/\S*)?/';
$replace = '<a href="$0">$0</a>';
$content = preg_replace($pattern, $replace, $content);
return $content;
}
public function collectData()
{
$category = $this->getInput('category');
$json = null;
if ($category == 'sotw') {
$url = static::URI . '/playlist/week';
$contents = getContents($url);
$shaderids = extractFromDelimiters($contents, 'var gShaderIDs = ', ';');
$shaderids = str_replace('\'', '"', $shaderids);
$url = static::URI . '/shadertoy';
$data = 's=' . rawurlencode('{ "shaders": ' . $shaderids . ' }') . '&nt=0&nl=0&np=0';
$header = [
'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:135.0) Gecko/20100101 Firefox/135.0',
'Content-Type: application/x-www-form-urlencoded',
'Accept: */*',
'Origin: https://www.shadertoy.com',
'Referer: https://www.shadertoy.com/playlist/week',
];
$opts = [
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $data,
CURLOPT_RETURNTRANSFER => true
];
$json = getContents($url, $header, $opts);
} else {
$url = static::URI . '/results?sort=' . $category;
$contents = getContents($url);
$json = extractFromDelimiters($contents, 'var gShaders=', 'var gUseScreenshots');
$json = substr(trim($json), 0, -1);
}
$json = Json::decode($json);
if (!$json) {
throw new Exception(sprintf('Unable to find css selector on `%s`', static::URI));
}
foreach ($json as $article) {
$id = $article['info']['id'];
$title = $article['info']['name'];
$author = $article['info']['username'];
$uri = static::URI . '/view/' . $id;
$content = '<p><img src="' . static::URI . '/media/shaders/' . $id . '.jpg"></p><p>' . $this->postprocessDescription($article['info']['description']) . '</p>';
$timestamp = $article['info']['date'];
$this->items[] = [
'title' => $title,
'author' => $author,
'uri' => $uri,
'content' => $content,
'timestamp' => $timestamp,
];
}
}
}

View File

@ -0,0 +1,209 @@
<?php
class SubstackProfileBridge extends BridgeAbstract
{
const NAME = 'Substack Profile';
const MAINTAINER = 'phantop';
const URI = 'https://substack.com/';
const DESCRIPTION = 'Returns posts from profiles on Substack';
const PARAMETERS = [[
'profile' => [
'name' => 'Profile name to use',
'exampleValue' => 'taliabhatt',
],
]];
private $name;
private $icon;
public function collectData()
{
$html = getSimpleHTMLDOMCached($this->getURI());
preg_match('/<script>window\._preloads\s*= JSON\.parse\("(.+?)"\)\s*<\/script>/', $html, $preg);
$json = stripcslashes($preg[1]);
$profile = json_decode($json, true)['profile'];
$this->name = $profile['name'];
$this->icon = $profile['photo_url'];
$id = $profile['id'];
$json = getContents(parent::getURI() . "api/v1/reader/feed/profile/$id");
foreach (json_decode($json, true)['items'] as $element) {
$this->items[] = $this->processAttachment($element);
}
}
private function processAttachment(array $element)
{
$item = [];
switch ($element['type']) {
case 'comment':
$element = $element['comment'];
$item['author'] = $element['name'] ?? $element['user']['name'];
$item['content'] = '';
if (isset($element['body_json'])) {
$item['content'] = $this->processBodyJson($element['body_json']);
}
$item['timestamp'] = $element['date'];
$item['title'] = 'Comment by ' . $item['author'];
$item['uri'] = $this->getURI() . '/note/c-' . $element['id'];
break;
case 'post':
$item['content'] = $element['postSelection']['text'] ?? '';
$element = $element['post'];
$item['author'] = $element['publishedBylines'][0]['name'];
$item['content'] .= $this->fetchPost($element['id']);
$item['timestamp'] = $element['post_date'];
$item['title'] = $element['title'];
$item['uri'] = parent::getURI() . 'home/post/p-' . $element['id'];
break;
case 'link':
$element = $element['linkMetadata'];
$item['author'] = $element['host'];
$item['content'] = $element['description'];
$item['title'] = $element['title'];
$item['uri'] = $element['url'];
break;
case 'image':
$item['uri'] = $element['imageUrl'];
break;
default:
throw new Exception('Invalid Substack entry type: ' . $element['type']);
}
$item['enclosures'] = [
$element['audio_items'][0]['audio_url'] ?? null,
$element['audio_items'][1]['audio_url'] ?? null,
$element['cover_image'] ?? null,
$element['image'] ?? null,
$element['imageUrl'] ?? null,
];
$item['categories'] = array_map(fn($tag) => $tag['name'], $element['postTags'] ?? []);
$item['comments'] = $item['uri'] . '/restacks/notes';
if (isset($element['attachments'])) {
foreach ($element['attachments'] as $attachment) {
$attachment = $this->processAttachment($attachment);
$item['categories'] = array_merge($item['categories'], $attachment['categories']);
$item['enclosures'] = array_merge($item['enclosures'], $attachment['enclosures']);
if (isset($attachment['title'])) { // Nothing to quote for images
$item['content'] .= $this->quoteAttachment($attachment);
}
}
}
return $item;
}
private function fetchPost(string $id)
{
$json = getContents(parent::getURI() . "api/v1/posts/by-id/$id");
$json = json_decode($json, true)['post'];
$html = str_get_html($json['body_html']);
$body = $html->root;
$block = $html->createElement('div');
$block->appendChild($html->createElement('hr'));
$block->appendChild($html->createElement('h4', 'Full text:'));
$block->appendChild($body);
return $block->innertext();
}
private function quoteAttachment(array $attachment)
{
$html = new simple_html_dom();
$body = $html->createElement('div');
$body->appendChild($html->createElement('hr'));
$link = $html->createElement('a');
$link->href = $attachment['uri'];
$link->appendChild($html->createElement('h3', $attachment['title']));
$body->appendChild($link);
if ($attachment['content'] != '') {
$body->appendChild($html->createElement('h4', 'Qouting ' . $attachment['author'] . ':'));
$body->appendChild($html->createElement('blockquote', $attachment['content']));
}
return $body->innertext();
}
private function processBodyJson(array $json)
{
$html = new simple_html_dom();
$body = $html->createElement('div');
foreach ($json['content'] as $block) {
if (isset($block['content'])) {
$content = $this->processBodyJson($block);
}
switch ($block['type']) {
case 'blockquote':
$content->tag = 'blockquote';
$body->appendChild($content);
break;
case 'paragraph':
$content->tag = 'p';
$body->appendChild($content);
break;
case 'text':
$text = $html->createTextNode($block['text']);
if (isset($block['marks'])) {
foreach ($block['marks'] as $mark) {
switch ($mark['type']) {
case 'bold':
$marked = $html->createElement('strong');
$marked->appendChild($text);
$text = $marked;
break;
case 'italic':
$marked = $html->createElement('em');
$marked->appendChild($text);
$text = $marked;
break;
case 'link':
$marked = $html->createElement('a');
$marked->href = $mark['attrs']['href'];
$marked->appendChild($text);
$text = $marked;
break;
default:
throw new Exception('Invalid text mark type: ' . $mark['type']);
}
}
}
$body->appendChild($text);
break;
case 'substack_mention':
$link = $html->createElement('a');
$link->href = parent::getURI() . 'profile/' . $block['attrs']['id'];
$link->appendChild($html->createTextNode($block['attrs']['label']));
$body->appendChild($link);
break;
default:
throw new Exception('Invalid body type: ' . $block['type']);
}
}
return $body;
}
public function getName()
{
$name = parent::getName();
if (isset($this->name)) {
$name .= " - $this->name";
}
return $name;
}
public function getIcon()
{
if (isset($this->icon)) {
return $this->icon;
}
return parent::getIcon();
}
public function getURI()
{
if ($this->getInput('profile') != null) {
return parent::getURI() . '@' . $this->getInput('profile');
}
return parent::getURI();
}
}

View File

@ -26,21 +26,16 @@ class TheFarSideBridge extends BridgeAbstract
$image = $card->find('img', 0);
$imageUrl = $image->attr['data-src'];
// Images are downloaded to bypass the hotlink protection.
$image = getContents($imageUrl, ['Referer: ' . self::URI]);
// Encode image as base64
$imageBase64 = base64_encode($image);
$caption = '';
if ($card->find('figcaption', 0)) {
$caption = $card->find('figcaption', 0)->innertext;
}
$item['enclosures'][] = $imageUrl;
$item['content'] .= <<<EOD
<figure>
<img title="{$caption}" src="data:image/jpeg;base64,{$imageBase64}"/>
<img title="{$caption}" src="{$imageUrl}"/>
<figcaption>{$caption}</figcaption>
</figure>
<br/>

View File

@ -57,6 +57,9 @@ class TldrTechBridge extends BridgeAbstract
continue;
}
$itemUrl = Url::fromString(self::URI . ltrim($child->href, '/'));
if ($itemUrl == $locationUrl) {
continue;
}
$this->extractItem($itemUrl);
if (count($this->items) >= $limit) {
break;
@ -125,6 +128,11 @@ class TldrTechBridge extends BridgeAbstract
}
}
}
foreach ($content->find('section') as $section) {
if (count($section->children()) == 0) {
$content->removeChild($section);
}
}
$title = $content->find('h2', 0);
return [$content->innertext, $title->plaintext];
}

View File

@ -0,0 +1,28 @@
<?php
class TomsToucheBridge extends BridgeAbstract
{
const NAME = 'Toms Touché';
const URI = 'https://taz.de/#!tom=tomdestages';
const DESCRIPTION = 'Your daily dose of Toms Touche.';
const MAINTAINER = 'latz';
const CACHE_TIMEOUT = 3600; // 1h
public function collectData()
{
$url = 'https://taz.de/';
$html = getSimpleHTMLDOM($url); // Docs: https://simplehtmldom.sourceforge.io/docs/1.9/index.html
$date = $html->find('p[x-ref]');
$date = trim($date[0]->innertext);
[$day, $month, $year] = explode('.', $date);
$image = $html->find('img[alt="tom des tages"]');
$item = [];
$item['title'] = "Toms Touché - $date";
$item['uri'] = 'https://taz.de/#!tom=tomdestages';
$item['timestamp'] = mktime(0, 0, 0, $month, $day, $year);
$item['content'] = $image[0] . '</img>'; // This isn't good HTML style, but at least syntactically correct
$item['uid'] = $image[0]->getAttribute('src');
$this->items[] = $item;
}
}

View File

@ -7,8 +7,8 @@ class VkBridge extends BridgeAbstract
// const MAINTAINER = 'ahiles3005';
const NAME = 'VK.com';
const URI = 'https://vk.com/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = 'Working with open pages';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Does not work anymore';
const PARAMETERS = [
[
'u' => [
@ -65,6 +65,7 @@ class VkBridge extends BridgeAbstract
public function collectData()
{
return;
$text_html = $this->getContents();
$text_html = iconv('windows-1251', 'utf-8//ignore', $text_html);
@ -391,10 +392,13 @@ class VkBridge extends BridgeAbstract
$item['categories'] = $hashtags;
// get post link
$post_link = $post->find('a.PostHeaderSubtitle__link', 0)->getAttribute('href');
preg_match('/wall-?\d+_(\d+)/', $post_link, $preg_match_result);
$item['post_id'] = intval($preg_match_result[1]);
$item['uri'] = $post_link;
$var = $post->find('a.PostHeaderSubtitle__link', 0);
if ($var) {
$post_link = $var->getAttribute('href');
preg_match('/wall-?\d+_(\d+)/', $post_link, $preg_match_result);
$item['post_id'] = intval($preg_match_result[1]);
$item['uri'] = $post_link;
}
$item['timestamp'] = $this->getTime($post);
$item['title'] = $this->getTitle($item['content']);
$item['author'] = $post_author;
@ -402,7 +406,7 @@ class VkBridge extends BridgeAbstract
// do not append it now
$pinned_post_item = $item;
} else {
$last_post_id = $item['post_id'];
$last_post_id = $item['post_id'] ?? null;
$this->items[] = $item;
}
}
@ -474,7 +478,10 @@ class VkBridge extends BridgeAbstract
if ($accurateDateElement) {
return $accurateDateElement->getAttribute('time');
} else {
$strdate = $post->find('time.PostHeaderSubtitle__item', 0)->plaintext;
$strdate = $post->find('time.PostHeaderSubtitle__item', 0)->plaintext ?? null;
if (!$strdate) {
return 0;
}
$strdate = preg_replace('/[\x00-\x1F\x7F-\xFF]/', ' ', $strdate);
$date = date_parse($strdate);

View File

@ -0,0 +1,86 @@
<?php
class YouTubeFeedExpanderBridge extends FeedExpander
{
const NAME = 'YouTube Feed Expander';
const MAINTAINER = 'phantop';
const URI = 'https://www.youtube.com/';
const DESCRIPTION = 'Returns the latest videos from a YouTube channel';
const PARAMETERS = [[
'channel' => [
'name' => 'Channel ID',
'required' => true,
// Example: vinesauce
'exampleValue' => 'UCzORJV8l3FWY4cFO8ot-F2w',
],
'embed' => [
'name' => 'Add embed to entry',
'type' => 'checkbox',
'required' => false,
'title' => 'Add embed to entry',
'defaultValue' => 'checked',
],
'embedurl' => [
'name' => 'Use embed page as entry url',
'type' => 'checkbox',
'required' => false,
'title' => 'Use embed page as entry url',
],
'nocookie' => [
'name' => 'Use nocookie embed page',
'type' => 'checkbox',
'required' => false,
'title' => 'Use nocookie embed page'
],
]];
public function getIcon()
{
if ($this->getInput('channel') != null) {
$html = getSimpleHTMLDOMCached($this->getURI());
$scriptRegex = '/var ytInitialData = (.*?);<\/script>/';
$result = preg_match($scriptRegex, $html, $matches);
if (isset($matches[1])) {
$json = json_decode($matches[1]);
return $json->metadata->channelMetadataRenderer->avatar->thumbnails[0]->url;
}
}
return parent::getIcon();
}
public function collectData()
{
$url = 'https://www.youtube.com/feeds/videos.xml?channel_id=' . $this->getInput('channel');
$this->collectExpandableDatas($url);
}
protected function parseItem(array $item)
{
$id = $item['yt']['videoId'];
$item['comments'] = $item['uri'] . '#comments';
$item['uid'] = $item['id'];
$thumbnail = sprintf('https://img.youtube.com/vi/%s/maxresdefault.jpg', $id);
$item['enclosures'] = [$thumbnail];
$item['content'] = $item['media']['group']['description'];
$item['content'] = str_replace("\n", '<br>', $item['content']);
unset($item['media']);
$embedURI = self::URI;
if ($this->getInput('nocookie')) {
$embedURI = 'https://www.youtube-nocookie.com/';
}
$embed = $embedURI . 'embed/' . $id;
if ($this->getInput('embed')) {
$iframe_fmt = '<iframe width="448" height="350" src="%s" title="%s" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>'; //phpcs:ignore
$iframe = sprintf($iframe_fmt, $embed, $item['title']) . '<br>';
$item['content'] = $iframe . $item['content'];
}
if ($this->getInput('embedurl')) {
$item['uri'] = $embed;
}
return $item;
}
}

View File

@ -75,7 +75,7 @@ custom_timeout = false
; "" = Disabled (default)
email = ""
; Advertise a contact Telegram url e.g. "https://t.me/elegantobjects"
; Advertise a contact URL (can be any URL!) e.g. "https://t.me/elegantobjects"
telegram = ""
; Show Donation information for bridges if available.

View File

@ -99,7 +99,7 @@ This method should return the HTML source as a base for the XPath expressions. U
This method should provide the feed title. Usually the XPath expression defined in `XPATH_EXPRESSION_FEED_TITLE` is used for extracting the title directly from the page source.
### Method `provideFeedIcon()`
This method should provide the feed title. Usually the XPath expression defined in `XPATH_EXPRESSION_FEED_ICON` is used for extracting the title directly from the page source.
This method should provide the URL of the feed's favicon. Usually the XPath expression defined in `XPATH_EXPRESSION_FEED_ICON` is used for extracting the title directly from the page source.
### Method `provideFeedItems()`
This method should provide the feed items. Usually the XPath expression defined in `XPATH_EXPRESSION_ITEM` is used for extracting the items from the page source. All other XPath expressions are applied on a per-item basis, item by item, and only on the item's contents.
@ -122,8 +122,8 @@ Accepts the items author as parameter, processes and returns it. Should return a
### Method `formatItemTimestamp()`
Accepts the items creation timestamp as parameter, processes and returns it. Should return a unix timestamp as integer.
### Method `cleanImageUrl()`
Method invoked for cleaning feed icon and item image URL's. Extracts the image URL from the passed parameter, stripping any additional content. Furthermore makes sure that relative image URL's get transformed to absolute ones.
### Method `cleanMediaUrl()`
Method invoked for cleaning feed icon, item image and media attachment (like .mp3, .webp) URL's. Extracts the media URL from the passed parameter, stripping any additional content. Furthermore, makes sure that relative media URL's get transformed to absolute ones.
### Method `fixEncoding()`
Only invoked when class constant `SETTING_FIX_ENCODING` is set to true. It then passes all extracted string values through PHP's `utf8_decode` function.

View File

@ -5,6 +5,11 @@ if (version_compare(\PHP_VERSION, '7.4.0') === -1) {
exit("RSS-Bridge requires minimum PHP version 7.4\n");
}
if (!extension_loaded('curl')) {
http_response_code(500);
exit("RSS-Bridge requires curl (apt install php-curl)\n");
}
require __DIR__ . '/lib/bootstrap.php';
require __DIR__ . '/lib/config.php';

View File

@ -240,7 +240,7 @@ abstract class BridgeAbstract
if (isset($input[$name])) {
$value = $input[$name];
} else {
if ($properties['type'] ?? null === 'checkbox') {
if (($properties['type'] ?? null) === 'checkbox') {
$value = false;
} elseif (isset($properties['defaultValue'])) {
$value = $properties['defaultValue'];
@ -327,7 +327,7 @@ abstract class BridgeAbstract
return $this->cache->get($this->getShortName() . '_' . $key, $default);
}
protected function saveCacheValue(string $key, $value, int $ttl = null)
protected function saveCacheValue(string $key, $value, int $ttl = 86400)
{
$this->cache->set($this->getShortName() . '_' . $key, $value, $ttl);
}

View File

@ -22,14 +22,7 @@ abstract class FeedExpander extends BridgeAbstract
if ($xmlString === '') {
throw new \Exception(sprintf('Unable to parse xml from `%s` because we got the empty string', $url), 10);
}
// prepare/massage the xml to make it more acceptable
$problematicStrings = [
'&nbsp;',
'&raquo;',
'&rsquo;',
];
$xmlString = str_replace($problematicStrings, '', $xmlString);
$xmlString = $this->prepareXml($xmlString);
$feedParser = new FeedParser();
try {
$this->feed = $feedParser->parseFeed($xmlString);
@ -59,6 +52,23 @@ abstract class FeedExpander extends BridgeAbstract
return $item;
}
/**
* Prepare XML document to make it more acceptable by the parser
* This method can be overriden by bridges to change this behavior
*
* @return string
*/
protected function prepareXml(string $xmlString): string
{
// Remove problematic escape sequences
$problematicStrings = [
'&nbsp;',
'&raquo;',
'&rsquo;',
];
return str_replace($problematicStrings, '', $xmlString);
}
public function getURI()
{
return $this->feed['uri'] ?? parent::getURI();

View File

@ -177,11 +177,9 @@ function getSimpleHTMLDOM(
}
/**
* Gets contents from the Internet as simplhtmldom object. Contents are cached
* Fetch contents from the Internet as simplhtmldom object. Contents are cached
* and re-used for subsequent calls until the cache duration elapsed.
*
* _Notice_: Cached contents are forcefully removed after 24 hours (86400 seconds).
*
* @param string $url The URL.
* @param int $ttl Cache duration in seconds.
* @param array $header (optional) A list of cURL header.

View File

@ -226,6 +226,63 @@ function defaultLinkTo($dom, $url)
return $dom;
}
/**
* Parse a srcset HTML attribute value and return size => URL mappings
* Srcset contains a list of image URLs with associated size specified as size (e.g. 1024w) or scale (e.g. 2x)
* The web browser should pick the most appropriate image depending on screen size and/or pixel density
*
* This function takes a srcset string such as the following:
* header640.png 640w, header960.png 960w, header1024.png 1024w
*
* Returns an array such as the following:
* [
* '640w' => 'header640.png',
* '960w' => 'header960.png',
* '1024w' => 'header1024.png'
* ]
*
* @param string $srcset Content of srcset html attribute
* @param bool $return_largest_url Instead of returning an array, return URL for the largest entry
* @return array|string Content of srcset attribute as { size => url } array, or largest entry URL if requested
*/
function parseSrcset(string $srcset, bool $return_largest_url = false)
{
// The srcset format is more tricky to parse that it seems:
// URLs may contain commas, and space after comma is not mandatory, so the following is valid:
// image.png?resize=640,640 640w,image.png?resize=960,960 960w,image.png?resize=1024,1024 1024w
// Since splitting by space or comma will not work, there is a precise algorithm to parse srcset attribute:
// https://html.spec.whatwg.org/multipage/images.html#parse-a-srcset-attribute
// To summarize, each srcset entry has the following format:
// 1. Leading spaces and comma. Zero or more spaces, zero or at most one comma
// 2. Any amount of characters up to the next whitespace (space, tab, newline...): This is the URL
// 3. A nonnegative number followed by lowercase w, x or h: This is the image size
// We parse the srcset entries using a regex to mimick the above parser/tokenizer behavior.
$preg_status = preg_match_all('/[\s]*,?[\s]*([^\s]+)\s+([0-9]+[wxh])/', $srcset, $matches);
$entries = [];
if ($preg_status !== false && $preg_status > 0) {
foreach ($matches[1] as $index => $url) {
if (array_key_exists($index, $matches[2])) {
$size = $matches[2][$index];
$entries[$size] = html_entity_decode($url);
}
}
}
if ($return_largest_url) {
$largest_image_url = null;
$largest_image_size = -1;
foreach ($entries as $size => $url) {
$size_int = intval(substr($size, 0, strlen($size) - 1));
if ($size_int > $largest_image_size) {
$largest_image_size = $size_int;
$largest_image_url = $url;
}
}
return $largest_image_url;
} else {
return $entries;
}
}
/**
* Convert lazy-loading images and frames (video embeds) into static elements
*
@ -244,28 +301,18 @@ function convertLazyLoading($dom)
$dom = str_get_html($dom);
}
// Retrieve image URL from srcset attribute
// https://developer.mozilla.org/en-US/docs/Web/API/HTMLImageElement/srcset
// Example: convert "header640.png 640w, header960.png 960w, header1024.png 1024w" to "header1024.png"
$srcset_to_src = function ($srcset) {
$sources = explode(',', $srcset);
$last_entry = trim($sources[array_key_last($sources)]);
$url = explode(' ', $last_entry)[0];
return $url;
};
// Process standalone images, embeds and picture sources
foreach ($dom->find('img, iframe, source') as $img) {
if (!empty($img->getAttribute('data-src'))) {
$img->src = $img->getAttribute('data-src');
} elseif (!empty($img->getAttribute('data-srcset'))) {
$img->src = $srcset_to_src($img->getAttribute('data-srcset'));
$img->src = parseSrcset($img->getAttribute('data-srcset'));
} elseif (!empty($img->getAttribute('data-lazy-src'))) {
$img->src = $img->getAttribute('data-lazy-src');
} elseif (!empty($img->getAttribute('data-orig-file'))) {
$img->src = $img->getAttribute('data-orig-file');
} elseif (!empty($img->getAttribute('srcset'))) {
$img->src = $srcset_to_src($img->getAttribute('srcset'));
$img->src = parseSrcset($img->getAttribute('srcset'));
} else {
continue; // Proceed to next element without removing attributes
}

View File

@ -175,24 +175,28 @@ function parse_mime_type($url)
'image' => 'image/*',
'mp3' => 'audio/mpeg',
];
// '@' is used to mute open_basedir warning, see issue #818
if (@is_readable('/etc/mime.types')) {
$file = fopen('/etc/mime.types', 'r');
while (($line = fgets($file)) !== false) {
$line = trim(preg_replace('/#.*/', '', $line));
if (!$line) {
continue;
}
$parts = preg_split('/\s+/', $line);
if (count($parts) == 1) {
continue;
}
$type = array_shift($parts);
foreach ($parts as $part) {
$mime[$part] = $type;
// if-check to avoid excessive php errors about open_basedir restriction (#4502)
$open_basedir = ini_get('open_basedir');
if (! $open_basedir) {
// '@' is used to mute open_basedir warning, see issue #818
if (@is_readable('/etc/mime.types')) {
$file = fopen('/etc/mime.types', 'r');
while (($line = fgets($file)) !== false) {
$line = trim(preg_replace('/#.*/', '', $line));
if (!$line) {
continue;
}
$parts = preg_split('/\s+/', $line);
if (count($parts) == 1) {
continue;
}
$type = array_shift($parts);
foreach ($parts as $part) {
$mime[$part] = $type;
}
}
fclose($file);
}
fclose($file);
}
}

View File

@ -52,7 +52,7 @@
<?php if ($admin_telegram): ?>
<div>
Telegram: <a href="<?= e($admin_telegram) ?>"><?= e($admin_telegram) ?></a>
Url: <a href="<?= e($admin_telegram) ?>"><?= e($admin_telegram) ?></a>
</div>
<?php endif; ?>

View File

@ -11,23 +11,23 @@ class RedditBridgeTest extends TestCase
$sut = new RedditBridge(new NullCache(), new NullLogger());
// https://old.reddit.com/search.json?q=cats dogs hen subreddit:php&sort=hot&include_over_18=on
$expected = 'https://old.reddit.com/search.json?q=cats+dogs+hen+subreddit%3Aphp&sort=hot&include_over_18=on';
$actual = RedditBridge::createUrl('cats,dogs hen', '', 'php', false, 'hot', 'single');
$expected = 'https://old.reddit.com/search.json?q=cats+dogs+hen+subreddit%3Aphp&sort=hot&include_over_18=on&t=all';
$actual = RedditBridge::createUrl('cats,dogs hen', '', 'php', false, 'hot', 'all', 'single');
$this->assertSame($expected, $actual);
// https://old.reddit.com/search.json?q=author:RavenousRandy&sort=hot&include_over_18=on
$expected = 'https://old.reddit.com/search.json?q=author%3ARavenousRandy&sort=hot&include_over_18=on';
$actual = RedditBridge::createUrl('', '', 'RavenousRandy', true, 'hot', 'user');
$expected = 'https://old.reddit.com/search.json?q=author%3ARavenousRandy&sort=hot&include_over_18=on&t=week';
$actual = RedditBridge::createUrl('', '', 'RavenousRandy', true, 'hot', 'week', 'user');
$this->assertSame($expected, $actual);
// https://old.reddit.com/search.json?q=cats dogs hen flair:"Proxy" subreddit:php&sort=hot&include_over_18=on
$expected = 'https://old.reddit.com/search.json?q=cats+dogs+hen+flair%3A%22Proxy%22+subreddit%3Aphp&sort=hot&include_over_18=on';
$actual = RedditBridge::createUrl('cats,dogs hen', 'Proxy', 'php', false, 'hot', 'single');
$expected = 'https://old.reddit.com/search.json?q=cats+dogs+hen+flair%3A%22Proxy%22+subreddit%3Aphp&sort=hot&include_over_18=on&t=month';
$actual = RedditBridge::createUrl('cats,dogs hen', 'Proxy', 'php', false, 'hot', 'month', 'single');
$this->assertSame($expected, $actual);
// https://old.reddit.com/search.json?q=cats dogs hen flair:"Proxy Linux Server" subreddit:php&sort=hot&include_over_18=on
$expected = 'https://old.reddit.com/search.json?q=cats+dogs+hen+flair%3A%22Proxy+Linux+Server%22+subreddit%3Aphp&sort=hot&include_over_18=on';
$actual = RedditBridge::createUrl('cats,dogs hen', 'Proxy,Linux Server', 'php', false, 'hot', 'single');
$expected = 'https://old.reddit.com/search.json?q=cats+dogs+hen+flair%3A%22Proxy+Linux+Server%22+subreddit%3Aphp&sort=hot&include_over_18=on&t=day';
$actual = RedditBridge::createUrl('cats,dogs hen', 'Proxy,Linux Server', 'php', false, 'hot', 'day', 'single');
$this->assertSame($expected, $actual);
}
}