What a missed marketing opportunity for Dlang... nevertheless, good news as sooner or later people will get it... and congrats to Weka.IO on "going public"...

https://www.theregister.co.uk/2017/07/13/wekaio_surfaces_after_swimming_submerged_against_the_current/

On Thursday, 3 August 2017 at 19:58:57 UTC, notna wrote:
> What a missed marketing opportunity for Dlang... nevertheless, good news as sooner or later people will get it... and congrats to Weka.IO on "going public"...
>
> https://www.theregister.co.uk/2017/07/13/wekaio_surfaces_after_swimming_submerged_against_the_current/

because , this is an article about the product weka sell , not an article about programming , an article written for users and buyers not programmers

had the article been written to target programmers, i am sure d would have been mentioned

On Thursday, 3 August 2017 at 19:58:57 UTC, notna wrote:
> What a missed marketing opportunity for Dlang... nevertheless, good news as sooner or later people will get it... and congrats to Weka.IO on "going public"...
>
> https://www.theregister.co.uk/2017/07/13/wekaio_surfaces_after_swimming_submerged_against_the_current/

Please tell me which enterprise storage company advertises the programming languages they implemented their product in. ;) We hope to have a post on the D blog with info from Weka sometime soon, that should be a good way to get the word out.

On Thursday, 3 August 2017 at 20:47:30 UTC, Joakim wrote:
> Please tell me which enterprise storage company advertises the programming languages they implemented their product in. ;) We hope to have a post on the D blog with info from Weka sometime soon, that should be a good way to get the word out.

They do mention it on their jobs page - http://www.weka.io/company/careers/
(see under "Data Path Developer" position).

On Thursday, 3 August 2017 at 20:55:35 UTC, Pradeep Gowda wrote:
> On Thursday, 3 August 2017 at 20:47:30 UTC, Joakim wrote:
>> Please tell me which enterprise storage company advertises the programming languages they implemented their product in. ;) We hope to have a post on the D blog with info from Weka sometime soon, that should be a good way to get the word out.
>
> They do mention it on their jobs page - http://www.weka.io/company/careers/
> (see under "Data Path Developer" position).

Sure, a lot of companies do that, not what I meant by advertising, ie in their news articles and press releases.  Weka has been very open by giving talks at DConf, that's going to be noticed more than some job opening buried in their website:

http://dconf.org/2015/talks/zvibel.html
http://dconf.org/2016/talks/zvibel.html

On Thursday, 3 August 2017 at 21:01:28 UTC, Joakim wrote:
> On Thursday, 3 August 2017 at 20:55:35 UTC, Pradeep Gowda wrote:
>> On Thursday, 3 August 2017 at 20:47:30 UTC, Joakim wrote:
>>> Please tell me which enterprise storage company advertises the programming languages they implemented their product in. ;) We hope to have a post on the D blog with info from Weka sometime soon, that should be a good way to get the word out.
>>
>> They do mention it on their jobs page - http://www.weka.io/company/careers/
>> (see under "Data Path Developer" position).
>
> Sure, a lot of companies do that, not what I meant by advertising, ie in their news articles and press releases.  Weka has been very open by giving talks at DConf, that's going to be noticed more than some job opening buried in their website:
>
> http://dconf.org/2015/talks/zvibel.html
> http://dconf.org/2016/talks/zvibel.html

Hi guys from Weka! Could you explain what difference from your product and Hadoop?

On 21/09/17 09:18, Suliman wrote:
> On Thursday, 3 August 2017 at 21:01:28 UTC, Joakim wrote:
>> On Thursday, 3 August 2017 at 20:55:35 UTC, Pradeep Gowda wrote:
>>> On Thursday, 3 August 2017 at 20:47:30 UTC, Joakim wrote:
>>>> Please tell me which enterprise storage company advertises the programming languages they implemented their product in. ;) We hope to have a post on the D blog with info from Weka sometime soon, that should be a good way to get the word out.
>>>
>>> They do mention it on their jobs page - http://www.weka.io/company/careers/
>>> (see under "Data Path Developer" position).
>>
>> Sure, a lot of companies do that, not what I meant by advertising, ie in their news articles and press releases. Weka has been very open by giving talks at DConf, that's going to be noticed more than some job opening buried in their website:
>>
>> http://dconf.org/2015/talks/zvibel.html
>> http://dconf.org/2016/talks/zvibel.html
> 
> Hi guys from Weka! Could you explain what difference from your product and Hadoop?

One is a linear database and the other is a filesystem?

If that doesn't satisfy you, please describe to me the difference between D and Microsoft Word, so I know what kind of answer you're expecting.

Shachar

On 22 September 2017 at 20:33, Shachar Shemesh via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 21/09/17 09:18, Suliman wrote:
>>
>> On Thursday, 3 August 2017 at 21:01:28 UTC, Joakim wrote:
>>>
>>> On Thursday, 3 August 2017 at 20:55:35 UTC, Pradeep Gowda wrote:
>>>>
>>>> On Thursday, 3 August 2017 at 20:47:30 UTC, Joakim wrote:
>>>>>
>>>>> Please tell me which enterprise storage company advertises the
>>>>> programming languages they implemented their product in. ;) We hope to have
>>>>> a post on the D blog with info from Weka sometime soon, that should be a
>>>>> good way to get the word out.
>>>>
>>>>
>>>> They do mention it on their jobs page -
>>>> http://www.weka.io/company/careers/
>>>> (see under "Data Path Developer" position).
>>>
>>>
>>> Sure, a lot of companies do that, not what I meant by advertising, ie in their news articles and press releases. Weka has been very open by giving talks at DConf, that's going to be noticed more than some job opening buried in their website:
>>>
>>> http://dconf.org/2015/talks/zvibel.html http://dconf.org/2016/talks/zvibel.html
>>
>>
>> Hi guys from Weka! Could you explain what difference from your product and Hadoop?
>
>
> One is a linear database and the other is a filesystem?
>
> If that doesn't satisfy you, please describe to me the difference between D and Microsoft Word, so I know what kind of answer you're expecting.
>

Haha. :-)

Indeed, Ceph would be closer to the mark for request of comparison. However their domain is more levelled as an alternative for S3 storage, and as block storage platform.  Their third offering (CephFS) is quite the shambles when it comes to performance.

Iain.

> One is a linear database and the other is a filesystem?
>
> If that doesn't satisfy you, please describe to me the difference between D and Microsoft Word, so I know what kind of answer you're expecting.
>

But Hadoop is more look like file system that DataBase...

On 23/09/17 11:57, Suliman wrote:
>> One is a linear database and the other is a filesystem?
>>
>> If that doesn't satisfy you, please describe to me the difference between D and Microsoft Word, so I know what kind of answer you're expecting.
>>
> 
> But Hadoop is more look like file system that DataBase...

Hadoop Distributed File System is, sort of, a file system. I don't know much about it (just read the Wikipedia page), so I'll try to answer as best I understand. Corrections welcome:

Performance:
I have not idea what HDFS's per-node performance numbers are, but there are several indications that make me suspect they are not as good as Weka's.

First of all, I don't think a tool written in Java, designed to run over another file system and the kernel's networking has any chance of out-performing a tool written in D, the directly uses the NVME and the network interface.

Second, the file system seems oriented toward large read-only blobs. As a file system, I don't think it has any chance against any dedicated Posix compliant file system, but I'm guessing you're mostly interested in using HDFS as a basis for running Hadoop itself, so that might not matter.


Cost:
Here I don't think there is any way for HDFS to compete. That might sound strange to some, as HDFS is open source while Weka charge licensing fees. The reason I'm saying this is because HDFS uses mirroring in order to achieve fault tolerance, while Weka uses Raid (I should know - I wrote it).

In short, to get 1PB of usable capacity while tolerating 2 faults you'll need 3PB of raw capacity with Hadoop (200% overhead). At 16+2, you'll only need around 1.3PB with Weka. Whatever you're paying for the licenses is, in all likelihood, going to be less than the cost of the hardware.


Like I said, corrections are welcome, as I'm not familiar with HDFS or Hadoop.

Shachar